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Preface 


Sensitivity  analysis  plays  an  important  role  in  the 
world  of  simulations  and  modelling.  Antiaircraft  artillery 
engagement  models  present  a special  type  of  problem  in  con- 
ducting sensitivity  analysis.  This  problem  is  due  to  the 
random  nature  of  the  many  variables  in  the  system.  I chose 
the  problem  addressed  in  this  study  because  the  nature  of 
problem  made  it  conducive  to  Monte  Carlo  sampling  techniques. 

The  organization  of  this  report  reflects  the  way  that 
I attacked  the  problem.  The  concept  of  the  analysis  was 
to  use  basic  physical  relationships,  as  much  as  possible, 
to  build  the  model  and  to  compute  data  points  from  the  model 
that  would  allow  the  required  sensitivity  analysis.  The 
Introduction,  Chapter  I,  and  Conclusions  and  Recommendations, 
Chapter  V,  outline  the  thrust  and  results  of  this  thesis. 
Chapters  II  and  IY  describe  the  analysis  methods  and  pro- 
cedures used,  and  Chapter  III  describes  the  model. 

My  sincere  thanks  and  deep  appreciation  must  be  ex- 
tended to  Captain  George  E.  Orr,  my  thesis  advisor,  for  his 
patience,  support,  and  assistance  in  this  effort.  I also 
want  to  thank  my  wife,  Rosie,  and  children  for  their  patience 
and  understanding. 


Errol  C.  Wilkins 


I 


Contents 


Preface  

List  of  Tables  

List  of  Figures  

Abstract  .........  . 

I.  Introduction  , 

AMRL's  Problem  , 

Thesis  Problem  Statement  . . . . , 

Objectives  , 

Scope  and  Limitations 

Assumptions  , 

Organization  , 

II.  Analysis  Methods  , 

P001  Methods 

Projectile  Trajectory  . . . 
Aircraft  Representation  . . 

Model  Development  ...  

Model  Concept  

Model  Variables  

Trajectory  Submodel  .... 
Aircraft  Representation  . . 
Monte  Carlo  Sampling  .... 
Encounter  Plane  Determination 
Analysis  Procedures  . 

III.  Simulation  Model  - AAASIM  


. Coordinate  Systems  ...  

Ground  Reference  System  

Aircraft  Reference  System  . . . . 
, Coordinate  Transformation  . , . . 

Shell  Trajectory  Calculations  

Calculation  of  Air  Density  . . . . 
Calculation  of  Drag  Coefficient 
Solution  of  Point  Mass  Equations  . 
Mean  Theoretical  Intercept  Point  . . . 
Monte  Carlo  Sampling  Procedures  . . . . 
Aiming  Angle  Distribution  . . . . 
Ballistic  Error  Sources  


ii 

vi 

vii 

viii 

1 


3 

4 

4 

5 

6 
7 

9 

9 

9 

. 11 
. 11 
. 12 
. 12 
. 13 
. 13 
. 13 
. 15 
. 15 

. 17 

. 18 
. 18 
. 20 
. 21 
. 22 
. 23 
. 23 
. 26 
. 27 
. 28 
. 28 
. 29 


..  ■*-  — if 


| 


Contents 


Page 

Probability  of  Hit  Calculations  30 

Function  Hit  .....  31 

Sample  Probability  of  Hit . 31 

Distribution  of  Shell  Impact  Points  32 

Encounter  Plane  Orientation  32 

Impact  Distribution  33 

Distribution  Testing  34 

Simulation  Output 34 

Analysis  and  Results 35 

Problem  Areas  .....  35 

Model  Variables 35 

Sample  Size  Considerations  . 36 

Typical  AAASIM  Run  Times 39 

Selective  Sampling  39 

Variable  Value  Selection  . 40 

Target  Model 40 

Aircraft  Flight  Path  Selection  41 

Time  of  Fire 42 

Elevation  Angles  43 

Intercept  Range  .....  44 

Ballistic  Error  Values  45 

Aiming  Angle  Distribution  Parameters  . . 45 

Sample  Size 47 

Results  of  Sensitivity  Analysis  48 

Sensitivity  Without  Ballistic  Errors  . . 49 

Sensitivity  With  Ballistic  Errors  ...  51 

Magnitude  of  Pk  Estimates 51 

Effect  of  Ballistic  Error  Values  ....  53 

Significance  of  Results 54- 

Confidence  Intervals 55 

P001  Response  Curves 57 

Bivariate  Normal  Assumption  57 

Gravity  Drop  Effects  .....  58 

Empirical  Trajectory  Effects  64 

Sensitivity  Comparison  65 

Conclusions  and  Recommendations  68 

Conclusions 69 

Probability  of  Kill  Sensitivity  ....  70 

Covariance  Effects  73 

Normality  Test 74 

Gravity  Affects 75 

Trajectory  Calculations  75 


iv 


1 


Figure 

3-1 

3-2 


4-1 

4-2 


4-3 

4-4 

4-5 


4-6 

4-7 


4-8 


List  of  Figures 


Page 

Depiction  of  Coordinate  Systems  . 19 

Drag  Force  Coefficient  vs  Mach  Number  .... 

23  ran  25 

Comparison  of  Sensitivity  Response  Curves 

With  and  Without  Ballistic  Error  Sources.  . . 50 

Affect  of  P^  Magnitude  on  P^.  Rate  of 

Change  With  Aiming  Angle  Variance 52 

Sample  AAASIM  Printout  Showing 

Pk  Confidence  Interval  56 

Normality  Test  of  Shell  Distribution 

Without  Ballistic  Error  Sources  59 

Shell  Distribution  Without  Gravity 

Effects 61 

Shell  Distribution  With  Gravity  Effects  ...  62 

Probability  of  Kill  vs  Aiming  Angle 

Variance  - 1500  meters  - Gravity 

Drop  Effects  63 

Probability  of  Kill  vs  Aiming  Angle 
Variance  - 1500  meters  - Comparison 
of  P001  and  AAASIM 66 


APIT/GST/MA/79M-6 


ABSTRACT 

This  study  was  conducted  to  investigate  the  sensitivity 
of  aircraft  probability  of  kill  (Pk)  estimates  to  the  para- 
meters of  a supplied  antiaircraft  artillery  aiming  angle 
distribution.  A model  was  developed  using  basic  physical 
relationships  for  the  shell  propagation  and  Monte  Carlo 
sampling  techniques  to  incorporate  the  random  errors  into 
the  model.  A shaped  target  was  used  and  all  random  error 
sources  were  assumed  to  be  bivariate  normally  distributed 
with  zero  means. 

Simulation  runs  were  made  holding  all  variables  constant 
except  for  the  aiming  angle  variances  and  response  curves 
were  constructed  by  plotting  versus  aiming  angle  variance. 
Tests  were  also  conducted  to  test  the  assumption  that  the 
shell  distribution  on  the  encounter  plane  was  bivariate 
normal  and  that  the  effects  of  gravity  drop  on  the  shell 
could  be  neglected.  The  results  from  the  study  model  were 
compared  with  simular  results  from  model  P001 . 

The  sensitivity  of  the  P^  estimates  was  found  to  be 
a function  of  the  magnitude  of  the  P^,  the  aiming  angle 
variance,  the  sum  of  the  ballistic  error  variances,  and  the 
aircraft  vulnerable  area.  Response  curve  trends  indicated 
slightly  more  sensitivity  from  the  study  model  than  from 
P001,  however,  it  could  not  be  concluded  with  certainty. 
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' SENSITIVITY  OP  AIRCRAFT  ATTRITION 
ESTIMATES  TO  AIMING  DISTRIBUTION 
PARAMETERS  OF  ANTIAIRCRAFT  ARTILLERY 

I Introduction 

In  recent  years  a number  of  studies  and  analyses  have 
been  conducted  in  which  the  expected  attrition  of  U.S.  air- 
craft by  enemy  antiaircraft  guns  has  been  evaluated  by  means 
of  mathematical  models.  The  three  most  prominent  models  are: 

Developing  Agency 

Army  Material  Systems  Analysis  Agency 
Aberdeen  Proving  Ground,  Maryland 

Air  Force  Armament  Laboratory 
Eglin  Air  Force  Base,  Florida 

Institute  for  Defense  Analyses 
Arlington,  Virginia 

The  main  purpose  of  these  models  is  to  provide  a method 
for  evaluating  optimal  tactics  and  countermeasures  to  be 
employed  by  penetrating  aircraft  into  hostile  environments. 
Antiaircraft  Artillery  models  can  be  divided  into  four 
functional  components  (or  submodels),  these  are:  (1)  deter- 
mination of  the  flight  path  of  the  aircraft,  (2)  determina- 
tion of  the  lead  azimuth  and  elevation  angles  of  the  gun, 
and  the  time  of  fire,  (3)  determination  of  the  trajectory  of 
the  fired  projectile,  (4)  calculation  of  the  probability  of 
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hit.  Simulation  of  these  four  areas  can  take  many  forms, 
from  purely  abstract  models  to  actual  engagements.  The 
degree  of  realism  modeled  will  greatly  affect  the  precision 
of  the  probability  of  hit  estimates  and  operational  useful- 
ness of  the  model(cost  of  running  the  model).  Antiaircraft 
engagement  models  attempt  to  trade-off  model  operational 
effectiveness  and  real  world  realism  by  making  simplifying 
assumptions  throughout  the  models.  These  assumptions,  which 
decrease  the  sophistication  of  the  models,  tend  to  smooth  out 
the  variations  in  the  system  while  increasing  the  operational 
effectiveness  of  the  model.  Smoothing  of  the  real  world  var- 
iations results  in  a distortion  of  the  sensitivity  of  the 
models  to  their  input  parameters  and  therefore  makes  it  dif- 
ficult to  determine  a particular  parameter’s  effect  on  the 
probability  of  hit  calculated  by  the  model. 

Currently  the  Analysis  and  Simulation  Branch  of  the 
6570th  Aerospace  Medical  Research  Laboratory  (AMRl/MEB)  at 
Wright-Patterson  APB,  Ohio  is  studying  the  effects  of  human 
trackers  in  the  man-machine  loop  of  an  antiaircraft  artillery 
and  aircraft  engagement.  The  thrust  of  AMRL's  study  is  to 
determine  if  some  tactics  and  countermeasures  make  it  more 
difficult  for  human  trackers  to  adequately  track  an  aircraft 
than  others.  To  ascertain  what  factors  influence  tracking 


performance,  AMRL  has  studied  the  different  actions  required 
of  trackers  and  have  developed  sophisticated  mathematical  and 
hardware  simulation  models  that  will  simulate  these  actions. 


AMRL's  Problem 

While  actual  measured  degradation  in  tracking  perfor- 
mance is  required  to  determine  overall  countermeasure  effec- 
tiveness, the  more  important  aspect  is  how  these  degradations 
affect  the  attrition  rates  to  he  expected  during  an  anti- 
aircraft artillery/penetrator  engagement.  This  affect  is 
being  measured,  at  AMR1,  by  incorporating  more  sophisticated 
tracking  submodels  into  the  Air  Force's  Antiaircraft  Artillery 
Simulation  Model  - P001 , and  using  P001  to  predict  the 
probability  of  kill. 

In  preliminary  work,  AMR1  has  observed  that  apparently 
significant  differences  in  tracking  performance  do  not  have 
as  much  impact  upon  attrition  estimates  as  might  be  expected. 
While  this  may  reflect  an  insensitivity  of  attrition  estimates 
to  tracking  errors,  it  is  also  possible  that  mismatches  in 
the  sophistication  and  precision  of  the  submodels,  down- 
stream of  the  tracking  submodel,  smooth  out  these  differ- 
ences. Two  areas  of  concern  have  been  identified  by  AMRL 
for  further  study,  these  are: 

1 . Lack  of  precision  in  the  terminal  effects 
model  used, 

2.  Differences  in  the  precision  and  sophisti- 
cation of  the  submodels  for  the  fire  control 
director,  gun  dynamics,  and  ballistics 
encounter,  especially  in  comparison  with  the 
tracking  models. 
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Thesis  Problem  Statement 
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Since  the  extensive  variations  in  tracking  performance 
measured  hy  AMRI  do  not  significantly  affect  the  overall 
aircraft  attrition  estimates  of  simulation  model  P001 , AMRI 
sponsored  this  thesis  to  obtain  a more  accurate  determina- 
tion of  the  sensitivity  of  aircraft  attrition  to  aiming 
distribution  parameters.  The  problem  to  be  studied  and 
analyzed  in  this  thesis  is:  How  sensitive  is  aircraft  attri- 
tion to  aiming  variances?  Specifically,  is  the  insensitivity 
of  aircraft  attrition  rates  to  significant  variations  in  the 
aiming  parameters  a result  of  mismatches  in  the  sophisti- 
cation of  the  submodels,  or  is  the  actual  probability  of  hit 
not  significantly  affected  by  these  variations  because  of 
other  perturbations  in  the  system? 

Objectives 

Intuitively,  one  would  expect  that  tracking  errors  would 
have  a definite  impact  on  aircraft  attrition.  Thus  it  is 
expected  that  a more  sophisticated  method  of  trajectory  and 
probability  of  hit  determination  will  reflect,  in  the  attri- 
tion estimates  produced,  the  significant  variations  in  track- 
ing performance  observed  by  AMRI.  Verification  of  this  ex- 
pected result  will  be  accomplished  by  completing  the  fol- 
lowing objectives: 

1 . Given  a distribution  of  gun  barrel  aiming  angles 
and  variances,  determine  the  sensitivity  of  air- 
craft attrition  to  this  distribution. 
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2.  Calculate  confidence  bands  for  the  estimates  of 
attrition  determined  in  objective  1. 

3.  P001  uses  the  assumption  that  the  shell  distri- 
bution on  the  encounter  plane  is  bivariate  normal; 
determine  if  this  assumption  is  valid. 

4.  Determine  if  the  affect  of  gravity  drop,  on  the 
shell  distribution,  is  significant. 

5.  Make  specific  recommendations  concerning  anti- 
aircraft artillery  terminal  effects  submodels  that 
would  increase  their  sensitivity  along  with  the 
operational  tradeoffs. 


Scone  and  Limitations 

The  results  of  AMRL's  tests  indicate  that  significant 
variations  in  tracking  performance  do  not  provide  equally 
significant  variations  in  attrition,  as  predicted  by  P001 . 

If  in  fact,  attrition  estimates  should  be  sensitive  to  track- 
ing performance,  there  are  three  major  submodels  where  some 
or  all  of  the  variations  could  be  smoothed:  the  fire  control 
director  submodel,  the  shell  trajectory  submodel,  and  the 
terminal  effects  submodel  (probability  of  hit  submodel). 
Losses  in  sensitivity  in  the  fire  control  director  are  the 
subject  of  another  study,  this  study  will  address  only  that  . 
portion  of  the  simulation  downstream  of  the  fire  control 
director.  That  is,  only  variations  in  aiming  angles,  muzzle 
velocities,  trajectory  disturbances,  and  methods  of  repre- 
senting the  aircraft  will  be  studied. 
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3.  Engagements  where  the  location  of  the  gun  or  the 
aircraft  is  not  known  with  certainty  will  not  he 
considered. 


Organization 

Chapter  II  provides  the  methods  used  in  the  study  along 
with  a brief  description  of  the  methods  of  P001 . The  methods 
used  in  P001  are  only  mentioned  as  a comparison  to  the 
methods  of  this  study. 

The  next  chapter,  Chapter  III,  will  describe  the  math- 
ematical model  used  by  this  study.  A complete  source  listing 
of  the  computer  code  is  contained  in  Appendix  A.  The  remain- 
ing chapters  will  discuss  the  results  of  the  analysis.  First, 
Chapter  IV  will  discuss  the  results  and  applications  of  the 
model  used  in  this  study,  simulation  run  printouts  support- 
ing this  analysis  will  be  found  in  the  text.  Second,  in 
Chapter  V this  work  will  be  summarized  and  conclusions  will 
be  presented. 
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II  Analysis  Methods 

The  first  phase  of  this  study  was  to  design  a model  to 
simulate  the  terminal  and  exterior  ballistics  of  an  anti- 
aircraft/aircraft engagement.  The  tacit  assumption  under- 
lying this  analysis  was  that  sensitivity  is  being  lost  in 
P001  between  the  tracking  submodel  and  the  final  probability 
of  kill  estimates.  Therefore,  the  logical  starting  point  was 
to  first  look  at  the  methods  which  are  used  in  model  POOI's 
exterior  and  terminal  ballistics  submodels.  Model  POO 1 was 
chosen  because  AMR1  is  using  that  model  to  predict  attrition 
estimate  variations  when  tracking  performance  is  degraded.  > 

P001  Methods 

P001  is  an  expected  value  model  that  evaluates  only  the 
means  and  variances  of  the  errors  associated  with  the  inputs 
to  the  fire  control  system.  The  model  then  computes  the  aim- 
point  and  calculates  the  resulting  probability  of  hit  by 
assuming  that  the  errors  are  independently  normally  dist- 
ributed. Two  aspects  of  the  modelling  methods  will  be  brief- 
ly considered  here:  they  are  the  method  of  modelling  the 
projectile  trajectory  and  the  method  of  representing  the  air- 
craft in  the  model.  A more  detailed  description  of  P001  can 
be  found  in  reference  4- . 

Projectile  Trajectory.  The  relationship  between  slant 
range  and  time  is  approximated  in  P001  by  an  empirical 
expression.  This  expression  assumes  a flat  trajectory 


(gravity  affects  are  not  considered),  and  is  given  by  the 
following  equation: 

R = vmt  / (1+at+bt2 ) (2-1) 

where:  R = slant  range 

t = time  of  flight  of  the  shell 
Vm  = muzzle  velocity (speed) 
a & b = empirically  derived  projectile  slowdown 
constants 

The  encounter  plane  is  assumed  to  be  normal  to  the  line 
of  sight  to  the  gun  through  the  point  where  the  mean  pro- 
jectile and  aircraft  are  equidistant  from  the  gun.  Once  the 
encounter  plane  has  been  determined,  the  shell  distribution 
on  this  plane  is  described  by  the  mean  aiming  point  and  a 
root-sum-square  conbination  of  the  aiming  angle  variances 
(which  are  assumed  to  be  independent),  and  six  other  sources 
of  random  error.  These  additional  sources  of  error  are: 
processing  error,  gun  jitter,  ballistic  dispersion,  atmos- 
pheric disturbance,  flight  roughness,  and  muzzle  speed 
uncertainty. 

All  of  the  error  sources  mentioned  above  are  included  in 
the  model  developed  for  this  study,  with  the  exception  of 
processing  errors.  Processing  errors  are  assumed  to  have 
been  previously  accounted  for  and  included  in  the  aiming 
angle  distribution  provided  for  this  study. 
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Aircraft  Representation.  The  target  aircraft  is  re- 
presented as  a diffused  target  bivariate  normally  distrib- 
uted across  the  encounter  plane.  The  diffused  target 
approximation  defines  a three-dimensional  Cartesian  co- 
ordinate system  in  which  the  origin  is  at  the  center  of  the 
aircraft,  and  the  Z-axis  points  in  the  direction  of  the  pro- 
jectile velocity  vector  at  the  moment  of  closest  approach  to 
the  target.  The  X and  Y axes  define  the  encounter  plane. 

This  type  of  damage  function  is  used  in  conjunction  with  the 
assumed  bivariate  normal  shell  distribution  so  that  the  prob- 
ability of  kill  can  be  determined  analytically.  Vulner- 
able areas  used  in  the  diffused  target  model  are  obtained 
from  a table  where  the  areas  are  expressed  as  a function  of 
relative  intercept  velocity. 

Model  Development 

The  Antiaircraft  Artillery  Simulation  Model  (AAASIM)  was 
developed  with  the  idea  of  using  more  sophisticated  methods 
than  those  used  in  P001 . A projectile  trajectory  submodel 
was  designed  that  used  point  mass  equations  of  motion.  The 
target  aircraft  was  modelled  as  a shaped  target,  and  Monte 
Carlo  sampling  techniques  were  used  to  incorporate  the  ran- 
dom errors  into  the  simulation.  In  making  these  changes  it 
was  realized  that  the  computer  run  time  for  the  model  would 
be  greater  than  P001 ; however,  it  was  hoped  that  a corre- 
sponding trade-off  would  be  realized  in  the  sensitivity  of 
the  attrition  estimates  to  aiming  angle  variances. 
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Model  Concept.  The  concept  behind  AAASIM  was  to  develop 
an  antiaircraft  artillery/ aircraft  engagement  model  that 
would  model  the  engagement  from  the  time  that  the  gun  was 
fired  until  the  shell  intercepted  the  encounter  plane;  or, 
either  hit  or  missed  the  aircraft.  A sample  run  of  this 
model  would  consist  of  specifying  the  system  independent 
variables  and  making  a number  of  replications  which  used 
Monte  Carlo  techniques  to  sample  the  random  errors  in  the 
system.  The  results  of  any  one  sample  run  would  be  either  a 
shell  distribution  on  the  encounter  plane  or  a probability 
of  hit  on  the  target  aircraft.  Thus,  by  varying  the  inputs 
to  the  model  and  observing  the  corresponding  probabilities  of 
hit,  the  sensitivity  of  the  probabilities  to  the  input  vari- 
ables could  be  determined. 

Model  Variables.  The  variables  exogenous  to  the  system 
consist  of  the  parameters  defining  the  aiming  angle  dis- 
tribution(aximuth  variance,  elevation  variance,  covariance, 
mean  aiming  angle  errors),  time  of  fire,  muzzle  velocity, 
projectile  trajectory  random  errors(gun  jitter,  ballistic 
dispersion,  atmospheric  disturbance , flight  roughness, 
muzzle  velocity  uncertainty),  aircraft  position  and  velocity, 
and  aircraft  vulnerable  area. 

The  variables  endogenous  to  the  system  consist  of  the 
slant  range  to  the  mean  intercept  point,  the  time  of  flight 
of  the  projectile  until  impact,  the  mean  azimuth  and  eleva- 
tion angles,  the  actual  azimuth  and  elevation  for  each 
replication,  the  actual  muzzle  velocity  of  the  shell  for 
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each  replication,  and  the  point  of  impact  on  the  encounter 
plane. 

Variables  input  into  the  model  that  are  related  to  the 
statistics  desired  are  the  random  number  starting  seed  and 
the  sample  size. 

Trajectory  Submodel.  One  of  the  sub-objectives  of  this 
study  was  to  determine  what  affect  the  gravity  drop  of  the 
shell  would  have  on  the  shell  distribution  on  the  encounter 
plane,  and  ultimately  on  the  attrition  estimates  produced. 
This  objective  was  accomplished  by  including  gravity  affects 
into  the  point  mass  equations  of  motion  of  the  shell.  These 
equations  are  described  in  Chapter  III.  A further  enhance- 
ment of  this  submodel  was  obtained  by  accounting  for  the 
varying  nature  of  the  aerodynamic  drag  coefficient  for  the 
shell,  the  air  density,  and  speed  of  sound  throughout  the 
flight  of  the  shell. 

Aircraft  Representation.  The  target  aircraft  is  repre- 
sented by  an  ellipsoid  whose  size  can  be  defined  by  input 
parameters.  This  ellipsoid  has  the  capability  of  moving  in 
three  dimensions.  The  use  of  a shaped  target  did  away  with 
the  necessity  to  calculate  the  projected  area  for  probability 
of  hit  calculations,  thus  enabling  the  calculations  to  be 
made  by  simply  counting  the  number  of  hits  and  dividing  by 
the  number  of  trials. 

Monte  Carlo  Sampling.  A mean  theoretical  intercept 
point (MTIP)  was  calculated  for  each  sample.  Prom  the  MTIP 
the  mean  aiming  azimuth  and  elevation  angles  could  be  deter- 
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mined.  Once  the  mean  aiming  angles  were  calculated,  each  of 
the  random  error  sources (except  muzzle  velocity  uncertainty 
and  flight  roughness)  could  be  sampled  using  Monte  Carlo 
techniques  and  the  resulting  random  errors  were  added  alge- 
braically to  the  mean  angles  to  arrive  at  the  aiming  angles 
for  a particular  replication.  Each  of  the  random  error 
sources  was  assumed  to  be  bivariate  normally  distributed 
with  a zero  mean. 

Muzzle  velocity  uncertainties  do  not  affect  the  aiming 
angles  directly,  but  rather  affect  the  time  of  flight  of  the 
shell  and  to  a lesser  extent  the  amount  of  gravity  drop. 
AAASIM  accounts  for  the  affects  of  muzzle  velocity  uncer- 
tainty by  keeping  the  aircraft  or  encounter  plane  moving 
throughout  the  engagement.  A slow  shell  will  take  longer  to 
reach  the  same  point  as  the  mean  shell,  during  this  extended 
time  the  aircraft  will  have  moved  further  along  the  flight 
path.  The  slow  shell  will  therefore  impact  the  aircraft  at 
a different  point  than  the  mean  shell,  or  miss  the  aircraft 
all  together.  The  same  logic  applies  to  a fast  shell,  only 
the  miss  distance  will  be  in  the  opposite  direction.  If  the 
model  is  being  used  to  determine  the  shell  distribution  of 
the  encounter  plane,  a moving  encounter  plane  will  result  in 
the  slow  or  fast  shell  impacting  the  plane  at  different  point 
than  the  mean  shell,  thus  altering  the  distribution. 

flight  roughness  is  the  error  source  associated  with  the 
uncertainty  of  the  aircraft  position  due  to  air  currents  and 
the  inability  of  the  aircraft  to  maintain  a constant  velocity, 
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from  the  time  the  shell  is  fired  until  impact.  Plight 
roughness  is  accounted  for  in  AAASIM  by  randomly  shifting 
the  aircraft  position  from  where  it  would  be  for  the  mean 
shell.  This  random  shifting  of  location  is  accomplished  by 
Monte  Carlo  Sampling  techniques. 

Encounter  Plane  Determination.  When  the  shell  dis- 
tribution is  desired  from  the  model,  the  encounter  plane  is 
positioned  perpendicular  to  the  relative  velocity  vector  and 
passes  through  the  moving  point  mass  representing  the  center 
of  the  aircraft.  Once  the  orientation  of  the  encounter  plane 
is  determined  for  the  mean  shell,  it  is  not  changed.  The 
plane,  however,  moves  throughout  the  engagement. 

A slight  error  is  induced  into  the  results  since  the 
orientation  of  the  encounter  plane  would  change  with  chang- 
ing shell  velocity.  This  effect  is  assumed  to  be  so  small 
in  comparison  with  the  magnitude  of  the  mean  shell  velocity 
as  to  be  insignificant. 

Analysis  Procedures 

The  number  of  different  combinations  of  variables  values 
in  this  model  is  infinite;  therefore,  to  make  the  analysis 
manageable  all  error  sources  were  fixed  in  value  except  for 
the  aiming  angle  variances  and  covariance.  Aircraft  velocity 
was  fixed  in  value  and  the  aircraft  position  was  adjusted  to 
provide  a constant  slant  range  from  the  gun  to  the  intercept 
point.  In  order  to  facilitate  the  comparison  between  dif- 
ferent samples,  the  same  random  number  generator  starting 
seed  was  used. 
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The  procedure  used  to  test  the  sensitivity  of  the  attri- 
tion estimates  to  the  aiming  angle  distribution  parameters 
consisted  of  the  following  steps.  The  aircraft  position  was 
adjusted  by  using  the  time  of  fire  so  that  the  point  of  mean 
intercept  was  where  the  shell  velocity  ve  :or  was  perpen- 
dicular to  the  longitudinal  axis  of  the  aircraft.  The  range 
to  the  intercept  point  was  also  kept  constant  for  a given 
test.  With  these  parameters  set,  a series  of  samples  were 
taken  varying  the  aiming  angle  variances.  Each  sample  run 
produced  a probabiltiy  of  hit  along  with  a confidence  band 
corresponding  to  the  sample  size. 

Another  series  of  test  were  made  using  the  same  pro- 
cedures described  above,  only  without  considering  the  grav- 
ity drop  of  the  shell.  This  series  was  used  to  determine  the 
affect  of  gravity  drop  on  probabilities  of  hit  produced  by 
the  model. 

The  final  series  of  test  conducted  were  made  to  provide 
the  shell  distribution  on  the  encounter  plane.  The  purpose 
of  these  runs  was  to  test  the  assumption  that  this  distri- 
bution was  bivariate  normal. 


This' section  has  provided  a discussion  of  the  methods 
of  the  analysis.  Chapter  III  is  devoted  to  presenting  a 
description  of  the  model  AAASIM  and  how  it  functions. 


Ill  Simulation.  Model  - AAASIM 


In  the  real  world  environment,  of  antiaircraft  artillery 
and  aircraft  engagements,  the  laws  of  nature  have  the  most 
pronounced  affect  on  whether  a given  artillery  shell  will 
hit  an  aircraft.  The  vulnerability  of  the  aircraft  will 
determine  whether  the  hit  will  disable  the  aircraft.  Speci- 
fically, once  the  gun  has  been  aimed  and  fired,  natural 
affects  cause  a dispersion  of  the  shell  from  its  mean  tra- 
jectory. These  affects  are  random  in  nature,  and  therefore, 
induce  an  uncertainty  as  to  whether  the  shell  and  aircraft 
will  intercept.  Besides  the  natural  random  affects,  the 
dynamics  of  the  engagement  affect  the  ability  of  human 
trackers  to  accurately  track  the  aircraft,  causing  a de- 
gradation in  tracking  performance.  This  degradation  is  re- 
flected in  the  aiming  angles,  thus  inducing  human  errors 
into  the  system.  A fortuitous  combination  of  these  errors, 
uncertainties,  and  perturbations  can  produce  a projectile 
trajectory  thich  will  intersect  that  of  the  aircraft. 

Assuming  something  is  known  about  the  way  in  which  each 
random  error  is  distributed,  a Monte  Carlo  sampling  can  be 
made  and  the  results  superimposed  upon  a mean  trajectory. 

Thus  the  basic  goal  of  this  simulation  model  is  to  mathe- 
matically characterize  the  total  projectile  trajectory, 
superimpose  the  known  random  errors  on  this  trajectory,  and 
dynamically  propagate  a projectile  until  a hit  or  miss 


situation  is  encountered. 

This  chapter  presents  a concise  discussion  of  the  model 
AAASIM.  The  concept  of  presentation  is  to  provide  the  reader 
with  a basic  understanding  of  the  methods  used  by  the  model. 
Explicit  detail  is  omitted  except  where  it  is  necessary  for 
clarification.  A complete  source  listing  of  the  computer 
code  is  contained  in  Appendix  A.  The  first  step  in  the  math- 
ematical characterization  is  to  define  the  coordinate  systems 
in  which  to  model  the  engagement. 

Coordinate  Systems 

Two  coordinate  systems  are  required  to  mathematically 
model  the  engagement.  The  primary  coordinate  system  is  the 
Ground  Reference  System(GRS).  The  gun  is  assumed  to  be  at 
the  origin  of  the  GRS.  The  secondary  coordinate  system  is 
the  Aircraft  Reference  System(ARS),  this  system  is  used  to 
define  the  shape  of  the  aircraft (see  figure  3-1). 

Ground  Reference  System.  The  ground  reference  system 
is  defined  by  three  axes;  X,  Y,  and  Z.  The  X-Y  plane  repre- 
sents the  horizontal  plane,  with  the  positive  Z axis  repre- 
senting the  "up"  vertical  direction. 

The  elevation  of  the  aircraft  from  the  gun  is  measured 
from  the  X-Y  plane  towards  the  positive  Z direction,  and  can 
take  on  values  of  0°  to  90°.  The  capability  exist  to  have 
negative  elevation  angles,  but  is  not  used  in  this  study. 

Azimuth  is  measured  from  the  positive  X axis,  counter- 
clockwise towards  the  positive  Y axis  and  can  take  on  values 


where:  9 is  the  azimuth  angle 

is  the  elevation  angle 

Figure  3-1 . Depiction  of  Coordinate 
Systems 
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from  0°  to  360°.  The  azimuth  and  elevation  angles  are  used 
to  locate  the  aircraft  reference  system  in  the  GRS.  These 
angles,  plus  the  range  from  the  origin  of  the  GRS  to  the 
origin  of  the  ARS,  positively  fix  the  ARS  in  the  GRS. 

Aircraft  Reference  System.  The  aircraft  reference 
system  is  a three-dimensional  coordinate  system  with  the 
origin  at  the  centroid  of  the  aircraft  model.  The  X’-axis 
is  coincident  with  the  longitudinal  axis  of  the  aircraft, 
with  the  positive  direction  toward  the  nose  of  the  aircraft. 
The  Y’-axis  passes  through  the  wings  of  the  aircraft  with 
the  positive  direction  towards  the  left  wing.  The  Z'-axis 
represents  the  vertical  direction(relative  to  the  aircraft), 
with  "up"  being  towards  the  top  of  the  aircraft. 

The  aircraft  is  represented  by  an  ellipsoid  specified  in 
terms  of  X',  Y* , and  Z*.  The  size  of  the  ellipsoid  can  be 
fixed  by  input  parameters.  As  stated  in  the  assumptions, 
of  Chapter  I,  the  ellipsoid  constitutes  the  vulnerable  area 
of  the  aircraft  and  a hit  anywhere  on  the  ellipsoid  consti- 
tutes a kill.  This  definition  of  "kill"  will  be  used 
throughout  the  remainder  of  this  report. 

The  position  of  the  aircraft  is  an  input  to  the  model 
as  a function  of  time.  The  entire  flight  path  of  the  air- 
craft must  be  input  as  straight (no  turns)  segments.  At  the 
start  of  each  segment  the  following  must  be  specified:  time, 
position  of  the  origin  of  the  ARS  in  the  X,  Y,  Z coordinates 
of  the  GRS,  velocity (actually  speed  in  this  case)  in  each  of 
the  three  directions,  and  roll  angle.  Prom  this  data, 
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Subroutine  ACPOS  interpolates  to  find,  the  position  of  the 
aircraft  at  any  time  required  during  the  simulation. 
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The  roll,  pitch,  and  yaw  of  the  aircraft  are  only 
reflected  in  the  GRS  since  the  ARS  is  fixed  to  the  aircraft. 
Positive  roll  is  defined  as  clockwise  rotation  of  the  ARS 
about  the  X’-axis(right  wing  down).  Yaw  is  rotation  of  the 
ARS  about  the  Z’-axis,  with  positive  being  a left  yaw.  Pitch 
is  defined  as  rotation  of  the  ARS  about  the  Y'-axis,  with 
positive  being  ”nose  up”.  The  roll  parameter  is  input  for 
each  flight  path  segment,  while  yaw  and  pitch  are  calculated 
by  ACPOS  using  the  velocity  components  of  the  aircraft. 

Coordinate  Transformation.  The  use  of  two  coordinate 
systems  necessitates  that  a transformation  be  used  so  that 
coordinates  in  one  system  can  be  expressed  as  coordinates  in 
the  other  system.  This  transformation  was  accomplished  with 
the  use  Euler  Angles.  Since  these  angles  do  not  provide  a 
unique  solution,  the  order  of  rotation  used  in  this  study  is: 
yaw,  pitch,  and  roll.  The  transformation  is  given  by: 
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(3-1) 


where : 


C and  S represent  Cosine  and  Sine  respectively 
Subscript  a represents  the  yaw  angle  - alpha 
Subscript  b represents  the  pitch  angle  - beta 
Subscript  g represents  the  roll  angle  - gamma 
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Subroutine  TRANS  computes  the  elements  of  the  transformation 
matrix  and  stores  them  for  later  use.  The  transformation  is 
used  by  the  program  to  transpose  the  shell  coordinates  in  the 
GRS  to  coordinates  in  the  ARS.  The  next  section  will  de- 

I scribe  the  propagation  of  the  shell  in  the  GRS. 

I Shell  Trajectory  Calculations 

The  shell  trajectory  calculated  by  AAASIM  uses  the 
theory  of  ballistic  trajectories  to  determine  the  shell 
position  as  a function  of  time.  In  the  calculation  of  bal- 
listic trajectories,  drag  is  normally  the  only  aerodynamic 
force  considered.  This  is  a valid  assumption  as  long  as  the 
angle  of  attack  of  the  shell  is  small.  Ballistic  projectiles 

Emay  have  an  angle  of  attack  when  they  exit  the  muzzle;  how- 
ever, for  spin  stabilized  ballistic  bodies  the  angle  of  at- 
tack will  be  near  zero  over  the  major  portion  of  the  tra- 
jectory. Thus,  forces  normal  to  the  axis  of  symmetry  can  be 
neglected  and  the  motion  of  the  shell  lies  in  a vertical  plane 
defined  by  the  initial  velocity  vector  and  the  gravity  vector. 

Point  mass  equations  of  motion  are  used  by  AAASIM  to 
calculate  the  position  of  the  shell  as  a function  of  time, 
these  equations  are: 


where:  X,  Y,  Z are  the  magnitudes  of  the  accelerations 

in  the  X,  Y,  Z directions  respectively. 

• • • 

X,  Yf  Z are  the  magnitudes  of  the  velocities  in 

the  X,  Y,  Z directions  respectively. 

V = the  magnitude  of  the  shell  velocity. 

R = the  density  of  the  air. 

A = the  cross-sectional  area  of  the  projectile 
perpendicular  to  the  axis  of  symmetry. 

G-p  = the  drag  coefficient. 

M = the  mass  of  the  projectile. 

G-  = the  magnitude  of  the  acceleration  due  to 
gravity. 

Calculation  of  Air  Density.  This  study  was  made  with 
the  assumption  that  the  effective  range  of  the  antiaircraft 
artillery  shell  would  he  no  greater  than  3000  meters.  Using 
this  assumption,  and  the  additional  assumption  that  the  gun 
was  at  sea  level,  the  air  density  (R)  was  approximated  hy 
the  following  linear  regression.  (Ref  6 :53): 

R = 1.220449545  - .000105481 1 (Z)  (3-6) 

where:  R = the  density  of  the  air  at  altitude  Z 

Calculation  of  Drag  Coefficient.  The  coefficient  of 
drag  is  an  experimentially  determined  parameter  that  varies 
with  the  mach  number  of  the  shell.  The  mach  number  of  the 
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shell  was  calculated  using  the  speed  of  sound  determined  in 
the  same  way  as  the  air  density.  The  linear  regressien 
used  for  the  speed  of  sound  is  (Ref 6 :53): 

S = 340.4616  - .003872727(Z)  (3-7) 

where:  S = the  speed  of  sound  at  altitude  Z 

Once  the  mach  number  of  the  shell  is  determined,  the 
appropiate  CD  can  he  extracted  Figure  3-2  which  was  taken 
from  Exterior  Ballistic  Data  For  Foreign  23mm  and  57  mm 
Antiaircraft  Systems  - HITVAL  I (Ref  i :26).  Projectile  par- 
meters  of  the  23mm  shell  were  used  exclusively  in  this  study. 
AAASIM  accounts  for  the  different  values  of  Cjj  in  the  sub- 
sonic, transonic,  and  supersonic  regions  by  calculating  the 
mach  number  of  the  shell  and  computing  CD  as  follows: 

CD  = .154  M < .86 

CD  = .300  .86  SMS  1.15 

CD  = .52689344  - .06590164(M)  M>>1.15 

where:  CD  = the  coefficient  of  drag  at  mach  number  M 

This  method  of  determining  induces  some  discontinuity; 
however,  it  is  expected  that  in  the  majority  of  cases  the 
speed  of  the  projectile  will  be  supersonic  causing  the 
effect  of  this  discontinuity  to  be  minimal. 
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The  values  for  the  cross-sectional  area  and  the  mass  of 


the  projectile  were  taken  from  the  HITVAD  I report(Ref  1 :11) 
and  were  held  constant  throughout  the  study.  The  values 
used  were: 


A = .0004154756  meters2 
m = .1885129  kg 

Solution  of  Point  Mass  Equations.  The  point  mass 
equations  of  motion  (equations  3-2,  3-3,  and  3-4)  do  not  have 
closed  form  solutions  because  of  their  non-linearity.  The 
equations  are  solved  in  AAASIM  numerically  by  a library  com- 
puter code  known  as  ODERT  (Ref  5 )•  This  code  solves  the 
initial  value  problem  for  ordinary  differential  equations  by 
predictor/corrector  methods.  Using  a start  time  and  a max- 
imum time,  this  code  integrates  from  the  start  time  in  the 
direction  of  the  maximum  time  until  it  locates  the  first  root 
of  a user  supplied  function.  Three  separate  user  defined 
functions  are  used  by  AAASIM  in  conjunction  with  ODERT  for 
different  segments  of  the  simulation,  these  will  be  discussed 
in  later  sections. 

Subroutine  ODERT  also  requires  the  user  to  supply  a 
function  subprogram  containing  the  differential  equations 
to  be  solved.  These  equations  are  contained  in  Function  F 
of  AAASIM  along  with  the  calculations  necessary  for  the 
aerodynamic  drag. 


The  first  step  in  the  simulation  is  to  determine  the 
mean  azimuth  and  elevation  aiming  angles  necessary  to  pro- 
vide an  intercept  between  the  projectile  and  aircraft  tra- 
jectories under  error  free  conditions. 


Mean  Theoretical  Intercept  Point 

The  mean  theoretical  intercept  point(MTIP)  is  defined 
as  the  intersection  of  the  projectile  and  aircraft  trajec- 
tories when  tracking,  aiming,  and  firing  are  done  in  an  error 
free  environment.  The  result  of  MTIP  calculations  is  the 
determination  of  the  mean  azimuth  and  elevation  angles  for 
the  gun  and  the  mean  time  of  flight  of  the  shell.  Once  these 
values  are  determined,  they  are  used  in  subsequent  proba- 
bility of  hit  calculations  and  shell  distribution  determina- 
tion. 


Subroutine  MTIP  is  the  routine  in  AAASIM  that  calcu- 
lates the  MTIP.  This  routine  uses  ODERT  and  the  user  sup- 
plied Function  DR  to  calculate  the  trajectory  of  the  shell. 
Function  DR  is  the  first  derivative  of  the  function  specify- 
ing the  distance  between  the  shell  and  the  centroid  of  the 
aircraft.  ODERT  integrates  outward  until  it  finds  the  first 
root  of  this  function  (the  point  where  the  distance  reaches 
a minimum) , it  then  returns  the  time  of  flight  of  the  shell 
and  the  coordinates  of  the  MTIP.  Subroutine  MTIP  uses  an 
iterative  procedure,  adjusting  the  aiming  angles,  until  the 
minimum  distance  between  the  shell  and  the  centroid  of  the 
aircraft  is  less  than  .05  meters.  When  this  test  is  passed, 
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the  aiming  angles  and  the  time  of  flight  of  the  shell,  to 
the  MTIP,  are  returned  to  the  calling  routine.  The  calling 
routine  can  now  superimpose  the  error  sources  of  the  system 
on  these  mean  aiming  angles  and  conduct  a Monte  Carlo  samp- 
ling to  calculate  either  the  probabiltiy  of  hit  or  the  shell 
distribution. 


Monte  Carlo  Sampling  Procedures 


AAASIM  uses  Monte  Carlo  techniques  to  incorporate  the 
identified  error  sources  into  the  simulation.  All  of  the 
error  sources  are  assumed  to  be  bivariate  normally  distri- 
buted with  zero  means.  This  study  was  also  made  assuming 
that  the  mean  aiming  angles  were  those  calculated  in 
Subroutine  MTIP.  Normal  random  deviates  used  in  the  error 
sampling  were  derived  in  Subroutine  GG-NOR , which  was  devel- 
oped using  the  Box-Mueller  transformation  to  transform 
uniform  random  deviates  into  normal  deviates.  The  uniform 
random  deviates  were  obtained  from  subroutine  GGUB,  a 
library  routine  in  the  International  Mathematical  and 
Statistical  Library  (IMSL).  Subroutine  G-GNRM,  also  from 
the  IMSL  library,  was  used  to  sample  from  the  aiming  angle 
covariance  matrix. (Ref  2 ). 

Aiming  Angle  Distribution.  Variance  in  the  aiming 
angles  associated  with  the  gun  barrel  are  input  to  the  model 
in  the  form  of  a covariance  matrix.  Monte  Carlo  samples 
are  taken  using  this  covariance  matrix  to  determine  random 
error  sources  for  the  aiming  angles.  These  errors  are  then 


Table  3-1 . Ballistic  Error  Values 


ERROR  SOURCE 

STD.  DEVIATION 

Muzzle  Velocity  Uncertainity 

10%  of  the  mean 

Atmospheric  Disturbance 

. 003  mils 

Gun  Jitter 

.005  mils 

Blight  Roughness 

(.iovaff) 

Ballistic  Dispersion 

.0031  mils 

(Ref  4 :28) 


superimposed  on  the  mean  aiming  angles.  The  next  step  is  to 
consider  the  ballistic  error  sources . 

Ballistic  Error  Sources.  There  are  several  other  sources 
of  error  that  affect  the  trajectory  of  the  shell,  these  are 
the  ballistic  error  sources.  These  errors  are  assumed  to  be 
bivariate  normal  with  zero  means  and  zero  covariance.  The 
ballistic  error  sources  considered  in  this  model  are: 

1.  Muzzle  velocity  uncertainties, 

2.  Atmospheric  disturbance, 

3.  Gun  jitter, 

4.  Blight  Roughness, 

5.  Ballistic  dispersion. 

These  error  sources  and  their  nominal  values  were  taken  from 
P001 , and  are  listed  in  Table  3-1 . The  impact  of  the  error 
sources,  and  their  values,  will  be  discussed  in  Chapter  IV. 


Plight  roughness  takes  into  account  the  random  move- 
ment of  the  aircraft  from  the  time  the  shell  is  fired  until 
the  shell  reaches  the  aircraft.  This  ballistic  error  source 
is  calculated  by  randomly  shifting  the  position  of  the  air- 
craft, from  the  position  determined  by  subroutine  ACPOS. 

The  amount  of  the  shift  is  determined  by  random  sampling 
a normal  distribution  with  zero  mean  and  a standard  deviation 
of  .010  (see  Table  3-1)  and  multiplying  the  value  obtained 
by  the  aircraft  speed  and  the  time  of  flight  of  the  shell. 

The  aircraft  position  is  shifted  in  three  independent 
directions  based  upon  its  speed  in  each  of  the  three  coor- 
dinate directions.  The  random  position  thus  obtained  is 
used  in  determining  the  probability  of  hit. 

Each  of  the  ballistic  error  sources  is  sampled  using 
Monte  Carlo  techniques  and  superimposed  upon  the  aiming  angls 
previously  determined  from  the  aiming  angle  distribution. 

Once  the  azimuth  and  elevation  angles  are  determined,  for  a 
particular  replication,  a shell  distribution  or  a probability 
of  hit  is  calculated  for  the  imput  sample  size. 

Probabiltiy  of  Hit  Calculations 

The  probability  of  hit  is  calculated  in  Subroutine 
SAMPLE  which  uses  Subroutine  SHOT  to  determine  whether  a 
particular  sample  replication  hits  the  aircraft,  or  not. 

The  azimuth  and  elevation  angles  of  the  shell,  with 
respect  to  the  GRS,  are  determined  in  Subroutine  SAMPLE  for 
each  replication  of  a particular  sample.  These  angles  are 


then  used  by  Subroutine  SHOT  to  propagate  a shell  toward  the 
aircraft  model.  Subroutine  ODERT  is  used  again  to  calculate 
the  trajectory  of  the  shell.  ODERT  integrates  from  the  time 
of  fire  until  it  finds  a root  of  the  user  defined  function, 
Function  HIT. 

Function  HIT.  The  general  equation  for  an  ellipsoid 
is: 


where:  a,  b,  c are  the  semi-axis  dimensions 

In  Function  HIT  the  right  hand  side  of  this  equation  is 
changed  to  a variable  and  the  derivative,  with  respect  to 
time  is  calculated.  For  each  time  interval  of  the  trajectory 
of  the  shell  calculated  by  ODERT,  the  coordinates  of  the  shell 
in  the  GRS  are  transformed  to  coordinates  in  the  ARS  using 
the  transformation  of  equation  3-1  and  substituted  into 
the  derivative  of  equation  3-8.  As  the  shell  comes  closer 
to  the  ellipsoid,  the  value  of  the  derivative  will  decrease 
and  as  the  shell  goes  further  from  the  ellipsoid  the  deriv- 
ative will  increase,  changing  sign  in  the  process.  At  the 
point  where  the  derivative  changes  sign,  if  the  value  of  the 
left  hand  side  of  eq.  3^8  equals  one  or  less,  a hit  is 
scored. 

Sample  Probability  of  Hit.  A tally  of  whether  each 
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sample  is  a hit  or  a miss  is  maintained  hy  Subroutine 
SAMPLE.  When  all  of  the  replications  have  been  run,  the 
sample  probability  of  hit  is  simply  the  total  number  of  hits 
divided  by  the  sample  size. 

After  the  probability  of  hit  is  calculated,  an  IMSL 
routine  BELBIN  is  used  to  calculate  confidence  bands  for 
the  estimated  probability  of  hit.  BELBIN  computes  a two- 
sided  confidence  interval  for  the  probability  of  hit  on 
any  trial  using  the  binominal  distribution. 

The  primary  output  from  this  option  of  the  simulation 
is  an  estimate  of  the  probabiltiy  of  hit  along  with  the 
confidence  interval  for  the  estimate.  Another  option  of 
AAASIM  is  the  determination  of  the  shell  impact  distri- 
bution on  the  encounter  plane. 

Listribiition  of  Shell  Impact  Points 

Simulation  model  AAASIM  has  the  capability  to  determine 
the  shell  distribution  on  the  encounter  plane  for  a partic- 
ular sample.  The  first  step  in  this  process  is  to  calculate 
the  orientation  of  the  encounter  plane. 

Encounter  Plane  Orientation.  The  encounter  plane  used 
in  this  study  is  a plane  perpendicular  to  the  relative  vel- 
ocity vector  of  the  shell,  with  respect  to  the  aircraft. 

The  plane  orientation  was  mathematically  simulated  by  using 
the  aircraft  reference  system,  disregarding  the  ellipdoidal 
aircraft  model. 

Realizing  that  a transformation  would  have  to  be  made 
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to  transform  the  coordinates  of  the  shell  location  into  the 
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encounter  plane,  the  transformation  developed  earlier 
(see  equation  3-1 ) could  be  used  if  pseudo  yaw  and  roll 
angles  could  be  determined.  These  pseudo  yaw  and  roll  angles 
are  the  azimuth  and  elevation  mean  aiming  angles  respective- 
ly. Using  the  mean  azimuth  and  elevation  aiming  angles,  the 
ARS  could  be  rotated  so  that  the  X'-Z’  plane  was  perpen- 
dicular to  the  relative  velocity  vector;  with  the  Y'-axis 
pointing  in  the  direction  of  the  relative  velocity  vector. 
Once  the  orientation  of  the  encounter  plane  is  determined, 
it  is  held  constant  throughout  a particular  sample. 

Impact  Distribution.  Subroutine  SAMPLE  uses  the  same 
procedures  as  were  used  for  probabiltiy  of  hit  calculations 
to  superimpose  the  identified  error  sources  upon  the  mean 
azimuth  and  elevation  aiming  angles.  Subroutine  DIST  then 
propagates  a particular  shell  along  its  trajectory,  using 
OLERT,  until  the  shell  impacts  the  encounter  plane.  This 
position  is  then  recorded  and  the  next  replication  is 
simulated. 

OLERT  uses  the  Function  YP  to  calculate  the  time  of 
encounter  plane  impact.  Function  YP  transposes  the  coor- 
dinates of  the  shell  into  ARS  coordinates  and  calculates 
the  Y*  coordinate  of  the  shell.  When  the  Y'  coordinate 
changes  sign  the  impact  point  on  the  X'-Z'  plane  can  be 
determined. 

It  is  recognized  that  shell  with  different  muzzle 
velocities  and/or  other  trajectory  errors  will  have  different 
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times  of  flight.  AAASIM  accounts  for  this  variation  by- 
moving  the  encounter  plane  along  the  aircraft  velocity 
vector  during  each  sample  engagement. 

Distribution  Testing.  Once  a sample  impact  distri- 
bution has  been  determined,  this  distribution  is  tested  to 
see  it  it  is  bivariate  normal.  The  first  step  in  this  test 
is  to  calculate  the  vertical  and  horizontal  means,  variances, 
and  the  covariance.  The  covariance  matrix,  thus  calculated, 
is  then  rotated  using  a Cholevsky  decomposition  to  obtain 
two  samples  that  can  be  independently  tested  for  normality. 
Each  of  these  samples  is  then  tested  for  normality  using  a 
Chi-square  goodness  of  fit  test. 

Simulation  Output.  The  simulation  outputs  from  the 
distribution  of  shell  impact  points  option  are  the  distri- 
bution parameters  and  the  goodness  of  fit  statistics.  Dis- 
tribution parameters  include:  the  vertical  and  horizontal 
means,  the  vertical  and  horizontal  variances,  the  covariance, 
and  the  correlation  coefficient.  The  goodness  of  fit  stat- 
istics include:  the  number  of  cells  used  in  the  test  (an 
input  option),  the  vertical  Chi-square  statistic,  the 
horizontal  Chi-square  statistic,  and  the  vertical  and  hori- 
zontal significant  levels. 

The  next  chapter  will  discuss  the  use  of  AAASIM,  analyze 
the  assumptions  used  in  the  modelling  process  and  discuss 
the  results  of  the  planned  simulation  runs. 
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The  purpose  of  this  study,  as  stated  earlier,  was  to 
determine  the  sensitivity  of  the  probability  of  hit  estimates 
to  the  parameters  of  the  aiming  angle  distribution.  The 
concept  of  development  for  AAASIM  was  to  develop  a model 
that  provided  a realistic  simulation  of  the  actual  external 
and  terminal  ballistics  of  an  antiaircraft  artillery  and 
aircraft  engagement.  The  resultant  model  provides  a more 
sophisticated  simulation  than  model  P001 , primarily  in  the 
ballistic  trajectory  and  target  model  portions.  Monte  Carlo 
techniques  were  used  to  superimpose  the  error  sources, 
(affecting  the  external  ballistics)  on  the  aiming  angle 
distribution  so  that  either  a shell  distribution  or  a proba- 
bility of  hit  could  be  determined.  The  use  of  Monte  Carlo 
techniques  induced  several  inherent  problems  into  the  analysis. 

Problem  Areas 

Simulations  using  Monte  Carlo  techniques  require  that 
a value  be  assigned  to  each  variable  of  the  model  and  that 
the  model  be  run  once  for  each  sample.  Since  each  model  run 
requires  a finite  amount  of  time  on  a computer,  effective  use 
of  the  computer  becomes  a major  concern.  The  two  major 
problems  that  have  to  be  dealt  with  are  the  number  of  vari- 
ables, and  the  sample  size  to  be  used. 

Model  Variables.  There  are  22  different  variables  con- 
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able  model  then  n samples  are  required  if  k parameters  are 
involved.  Since  each  of  the  variables,  considered  in  AAASIM, 
is  continuous  in  nature  the  number  of  model  runs  can  quickly 
become  overwhelming.  To  complicate  the  matter  further,  the 
number  of  samples  for  each  model  run  must  be  considered. 

Sample  Size  Considerations.  The  accuracy  of  point 
estimates  for  probabilistic  variables  is  highly  dependent 
upon  sample  size.  If  the  point  estimate  desired  is  the  proba- 
bility of  hit,  the  sample  size  can  be  estimated  by  approxi- 
mating the  binomial  distribution  with  a normal  distribution. 
This  approximation  is  acceptable  if  the  sample  size  is  large 
and  the  actual  probability  of  hit  is  not  near  zero  or  one. 

The  equation  used  (Ref  3:131)  is: 


is  the  required  sample  size 

is  the  two-tailed  standardized  normal  statistic 
for  the  probability  desired 
is  the  significance  level  desired 
i3  the  largest  acceptable  difference  between 
the  estimate  and  the  true  probability 

Table  4-1  contains  the  approximate  number  of  samples  re- 
quired for  different  levels  of  significance  and  different 

d’s. 


where : n - 

Za/2  " 

a - 
d - 
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Table  4-1 . Minimum  Sample  Size  For  Bernoulli  Estimates 


r 


a 

Za/2 

d 

n 

.10 

1.65 

.1000 

68 

.10 

1.65 

.0100 

6806 

.10 

1.65 

.0010 

680625 

.10 

1.65 

.0001 

68062500 

.05 

1.96 

.1000 

96 

ir\ 

O 

• 

1.96 

.0100 

9604 

.05 

1.96 

.0010 

960400 

.05 

1.96 

.0001 

96040000 

.01 

2.58 

.1000 

166 

.01 

2.58 

.0100 

16641 

.01 

2.58 

.0010 

1664100 

.01 

2.58 

.0001 

166410000 

The  sample  sizes  shown  in  Table  4-1  are  valid  when  the 
actual  probability  of  hit  is  not  near  zero.  According  to 
simulation  runs,  with  zero  mean  aiming  errors,  this  situation 
occurs  when  the  range  of  the  intercept  is  less  than  1000 
meters  and  the  vulnerable  area  is  reasonable  value,  i.e. 
in  the  order  of  5-7  meters2.  For  most  engagements,  however, 
the  range  will  be  in  the  order  of  1000  - 2000  meters;  in  this 
range  the  probabilities  of  hit  are  typically  on  the  order  of 
.001  - .01.  In  this  range  of  probabilities  the  sample  sizes 
in  Table  4-1  are  somewhat  conservative.  A more  realistic 
sample  size  for  three  significant  figures  is  approximately 
10000.  Table  4-2  provides  some  typical  ranges  for  accuracy 
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Table  4-2.  Confidence  Interval  Bounds  For 
95%  Confidence  level 


n 

Pk 

Upper  limit 

lower  limit 

1 9G0 

.001 

.0056 

.0000 

1000 

.010 

.0183 

.0048 

1000 

.100 

.1202 

.0821 

5000 

.001 

.0023 

.0003 

5000 

.010 

.0132 

.0074 

5000 

.100 

.1087 

.0918 

10000 

.001 

.0018 

.0005 

10000 

.010 

.0121 

.0081 

10000 

.100 

.1060 

.0942 

of  predictions  that  were  calculated  from  the  binominal 
distribution  using  an  IM51  library  routine,  BELBIN. 

The  conclusion  to  be  drawn  from  these  two  tables  is 
that  for  low  single  shot  probabilities  of  hit,  extreme 
accuracy  is  only  possible  with  very  large  sample  sizes,  lack 
of  accuracy  can  be  detrimental  when  trying  to  distinguish 
between  probabilities  of  hit  that  are  close  in  value.  It 
also  means  that  to  use  computed  values  to  several  significant 
figures  can  be  misleading. 

The  large  sample  sizes  that  are  required  for  reasonable 
accuracy  coupled  with  the  vast  number  of  possible  variable 
values  necessitated  that  restrictions  be  placed  on  the  allow- 
able range  of  the  variables,  and  the  sample  sizes  considered. 
Without  these  restrictions  the  computation  time  would  have 
been  unacceptable. 
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Typical  AAASIM  Run  Times,  The  model  designed  for  this 
study,  AAASIM,  was  not  necessarily  designed  for  efficiency. 
However,  when  it  was  expedient  to  do  so,  time  saving  steps 
were  taken.  The  simulation  run  time  varied  with  the  range 
to  the  intercept  point,  the  option  run,  and  the  sample  size. 
Typical  run  times  for  AAASIM  were  on  the  order  of  40  - 70 
seconds.  When  the  run  option  desired  is  the  computed  proba- 
bility of  hit,  AAASIM  uses  a method  of  selective  sampling 
to  lower  the  run  time. 

Selective  Sampling.  When  the  probability  of  hit  is  de- 
sired, each  shell  is  propagated  from  firing  until  the  shell 
either  intercepts  or  misses  the  target.  Once  the  azimuth  and 
elevation  angles  of  propagation  are  determined  (by  super- 
imposing each  of  the  error  sources  upon  the  input  aiming 
angles)  it  can  be  analytically  determined  if  the  shell  has 
any  possibility  of  intercepting  the  target. 

Since  this  model  uses  a shaped  target,  if  the  target 
position  is  known  with  certainty  the  firing  angle  necessary 
to  just  miss  the  target  can  be  calculated.  To  account  for 
different  target  positions,  due  to  different  shells  times  of 
flight,,  the  semi-axes  of  the  ellipsoidal  shaped  target  are 
enlarged  by  three  meters.  The  critical  firing  angle  is  then 
calculated  analytically  and  used  for  a cut-off  point  for 
propagating  shells.  If  the  sample  aiming  angles  are  out- 
side of  these  computed  critical  values  the  shell  is  not 
propagated,  a miss  is  scored  and  the  next  sample  is  run. 
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This  step  bypasses  the  most  time  consuming  portion  of  the 
simulation,  thus  saving  the  computation  time  in  cases  where 
the  probability  of  a hit  is  zero. 

In  contrast,  the  simulation  option  that  computes  the 
shell  distribution  on  the  encounter  plane  must  propagate 
each  shell  through  to  intercept,  regardless  of  nearness  of 
the  impact  point  to  the  target.  This  option  will  always 
require  the  longest  time  to  run. 

The  method  of  selective  sampling  used  provides  one 
computation  time  saving  device;  however,  the  major  time 
saving  device  employed  in  the  study  was  to  reduce  the 
variables  considered  and  therefore  reducing  the  replications. 

Variable  Value  Selection 

In  order  to  make  the  study  manageable,  and  to  con- 
centrate on  the  sensitivity  analysis  desired,  a number  of 
variables  will  be  discussed  in  this  section  along  with  the 
possible  ramifications  of  holding  them  constant. 

Target  Model.  The  target  aircraft  was  modelled  as  a 
shaped  target  throughout  the  study.  This  is  in  contrast  to 
modelling  the  target  as  a damage  function,  i.e.  POOI's  dif- 
fused target  model.  The  selection  of  a shaped  target  makes 
it  possible  to  model  the  probability  of  hit  as  a series  of 
Bernoulli  trials,  and  one  shot  is  either  a hit  or  miss.  One 
advantage  of  this  method  of  target  representation  should  be 
obvious,  it  reduces  the  computation  of  the  probability  of 
hit  to  simply  scoring  hits  and  misses.  A second  advantage 
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is  the  target  can  he  moving  throughout  the  simulated  engage- 
ment, thus  automatically  accounting  for  differences  in  shell 
velocities  and  actual  aircraft  movement. 

The  disadvantage  of  this  type  of  target  model  is  that 
it  is  difficult  to  accurately  represent  the  geometric  shape 
of  an  aircraft.  The  shape  used  in  AAASIM  is  an  ellipsoid, 
and  hy  selectively  defining  the  dimensions  of  the  semi-axes, 
the  shape  fairly  accurately  represents  the  body  of  an  aircraft. 

The  use  of  this  model  carries  with  it  the  implicit  assumption 
that  the  wings,  elevator,  and  vertical  stablizer  are  not  part 
of  the  aircraft  vulnerable  area.  The  vulnerable  area  of  such 
a model  can  be  controlled  by  varying  the  semi-axes  dimensions. 

This  was  done  in  later  comparisons  of  AAASIM  results  with 
those  of  P001 . 

A constant  presented  area,  even  though  not  necessarily 
accurate,  is  a desirable  property  when  measuring  the  sen- 
sitivity of  probability  of  hit  to  aiming  angle  distributions. 

A constant  presented  area  is  obtained  by  fixing  the  flight 
path  of  the  aircraft  and  time  to  fire. 

Aircraft  Plight  Path  Selection.  The  aircraft  flight 
path  was  kept  straight  and  level  for  all  replications  of  the 
simulation.  The  aircraft  track  was  set  parallel  to  the  X-axis 
of  the  ground  reference  system,  and  the  aircraft  had  a veloci- 
ty only  in  the  X direction.  The  Y-axis  displacement  was 
adjusted  along  with  the  Z - axis  displacement  to  provide  the 
desired  range  and  elevation  for  a particular  replication. 

Using  a common  start  point,  the  location  of  the  aircraft  at 


41 


the  time  of  intercept  could  he  controlled  by  the  time  of  fire. 

Time  of  fire.  As  mentioned  earlier,  a constant  aircraft 
presented  area  is  a desirable  property  when  looking  at  the 
effect  of  varying  the  aiming  angle  distribution.  This  was 
accomplished  in  AAASIM  by  adjusting  the  time  of  fire  so  that 
the  mean  shell  intercepted  the  target  aircraft  perpendicular 
to  the  longitudinal  axis  of  the  aircraft.  In  the  model  geo- 
metry, this  point  is  defined  as  an  azimuth  aiming  angle  of 
90°.  When  the  intercept  point  is  so  located,  the  presented 
area  of  the  target  is  not  a function  of  the  elevation  angle; 
since  with  the  ellipsoid  model  any  elevation  will  have  the 
same  presented  area  as  long  as  the  cross-sectional  area 
perpendicular  to  the  longitudinal  axis  is  circular.  This  is 
also  the  point  of  closest  approach  of  the  aircraft  to  the  gun, 
and  the  point  where  the  azimuth  tracking  rate  is  the  greatest. 

Modelling  the  engagement  at  the  point  where  the  azimuth 
tracking  rate  is  the  greatest  might,  at  first,  seen  counter- 
productive;, however,  this  study  does  not  model  the  tracking 
explicitly  but  assumes  a given  aiming  angle  distribution. 
Therefore,  the  errors  that  this  method  might  induce  in  the 
tracking  performance  are  not  relevant  to  this  study  and  are 
ignored. 

A slight  error  is  induced  into  the  computations  be- 
cause the  actual  shell  velocities  vary  about  the  mean  shell 
velocity,  and  all  shells  will  not  see  exactly  the  same  pre- 
sented area.  This  error  is  assumed  to  be  very  small  and  ig- 
nored. 
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Elevation  Angles.  There  are  an  infinite  number  of  pos- 
sible elevation  angles  that  could  be  considered.  Since  it 
would  be  impossible  to  consider  all  possible  values,  some 
restriction  had  to  be  placed  on  the  elevation  angle  that  would 
be  used  in  the  simulation,  in  the  previous  sections  the 
location  of  the  target  aircraft  had  been  so  positioned  that 
the  presented  area  would  not  be  a function  of  the  firing 
elevation  angle.  The  remaining  consideration  for  the  selec- 
tion of  possible  elevation  angles  was  the  interrelation  bet- 
ween elevation  angle  and  the  gravity  drop  of  the  projectile. 

One  of  the  objectives  of  this  study  was  to  determine  the 
effect  of  gravity  drop  on  the  shell  distribution.  The  pos- 
sibility existed  that  gravity  drop  would  distort  the  shell 
distribution  and  thus  alter  the  probability  of  hit.  Intui- 
tively, the  effects  of  gravity  would  be  zero  on  the  direction 
of  a shell  that  traveled  in  the  vertical  direction,  and  would 
be  the  greatest  on  a shell  that  traveled  in  the  horizontal 
direction.  Therefore,  an  elevation  angle  of  zero  was  select- 
ed so  that  the  simulation  results  would  yield  a worst  case 
situation  of  the  effects  of  gravity  on  the  shell  distribution. 

The  actual  situation  modelled  was  to  position  the  air- 
craft so  that  the  elevation  angle  of  the  line  of  sight  from 
the  gun  to  the  aircraft  was  zero.  To  accomplish  this,  the 
actual  aiming  angle  had  to  be  elevated  slightly  to  compensate 
for  the  gravity  drop.  This  was  strictly  a modelling  con- 
sideration and  the  error  induced  should  be  negligible. 
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Intercept  Range.  The  next  possibility  considered  to 
reduce  the  range  of  variables  was  to  limit  the  intercept 
ranges  that  would  be  used  in  the  simulations.  Range  has 
two  primary  effects  on  probability  of  hit  estimates.  The 
first  is  that  as  the  range  of  the  shell  increases,  its  speed 
decreases  due  to  drag  forces.  As  the  speed  of  the  shell 
decreases,  its  ability  to  damage  the  target  (considering 
that  it  does  hit  the  target)  also  decreases.  Below  some 
minimum  value  the  shell  is  no  longer  a threat  to  the  target 
aircraft.  The  second  effect  of  range  is  that  trajectory 
perturbations  tend  to  cause  a dispersion  in  the  shell 
position  from  the  mean  shell  position.  Thus  as  the  range 
increases  the  probability  of  hitting  the  target  decreases. 

These  two  effects  of  range  combine  to  provide  a pract- 
ical maximum  limit  of  the  ranges  to  be  considered.  As  far 
as  a minimum  range  limit,  the  practical  considerations  are 
the  altitude  of  the  aircraft  and  the  fact  that  as  the  inter- 
cept range  gets  extremely  small(less  than  500  meters),  the 
error  effects  are  negligible,  resulting  in  very  high  proba- 
bilities of  hit.  The  range  of  1500  meters  was  chosen  to 
make  the  most  of  the  comparisons  because  it  provided  a com- 
promise that  allowed  trajectory  perturbations  to  have  an 
effect  on  the  trajectory  and  still  provide  probabilities  of 
hit  estimates  that  gives  an  appreciable  number  of  expected 
hits  in  the  number  of  shots  sampled.  Appreciable  refers  to 
earlier  remarks  on  confidence  bounds. 
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Ballistic  Error  Values.  The  values  for  the  ballistic 
error  sources  considered  in  this  study  are  standard  devia- 
tions of  circular  normal  distributions.  Each  error  source  is 
assumed  to  be  circular  normally  distributed  with  a zero  mean 
and  a zero  covariance.  The  error  sources  are  also  assumed  to 
be  independent  of  each  other.  The  standard  deviations  used 
were  the  values  used  in  P001  and  were  held  constant  through- 
out the  study.  Their  values  are  shown  in  Table  3-1 . 

The  ballistic  error  sources,  as  well  as  their  values, 
are  a major  source  of  uncertainty  in  this  analysis.  The 
effects  of  this  uncertainty  will  be  discussed  in  later  sect- 
ions of  this  chapter. 

Aiming  Angle  Distribution  Parameters.  The  aiming  angle 
distribution  is  assumed  to  be  bivariate  normal  and  defined 
by  a covariance  matrix  and  mean  values  of  zero.  The  covar- 
iance matrix  contains  the  remaining  variables  to  be  con- 
sidered in  the  simulation.  It  is  composed  of  a variance  for 
the  azimuth  aiming  angle,  a variance  for  the  elevation  aim- 
ing angle,  and  a covariance.  The  sensitivity  that  is  the 
subject  of  this  study  is  the  sensitivity  of  Pk  estimates  to 
these  variances  and  covariance. 

AMRIj  has  modified  P001  to  account  for  more  sophisticated 
determination  of  tracking  performance.  Using  AMRIi’s  version 
of  P001 , typical  values  of  the  aiming  angle  variances  were 
computed.  These  values  were  in  the  range  of  .000001  to 
.000200.  Assuming  that  this  range  is  typical  of  the  var- 
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iances  that  could  be  expected,  this  was  the  range  used  in. 
this  study.  In  order  to  reduce  the  number  of  simulation 
replications  required,  the  distribution  was  assumed  to  be 
circular  normal  (both  variances  equal  for  each  replication). 
In  reality  the  variances  would  very  seldom  be  equal,  and  if 
it  is  assumed  that  most  aircraft  flight  paths  would  be  more 
horizontal  than  vertical,  a slight  error  will  be  induced  in 
the  vertical  component  of  the  shell  distribution  calculated. 

The  effect  of  this  vertical  error  should  be  to  under- 
estimate the  calculated  probability  of  hit.  Since  this 
study  is  concerned  with  relative  values,  rather  than  ab- 
solute values,  the  effect  of  the  error  on  the  results  of  the 
study  should  be  minimal . 

The  covariance  effect  was  not  readily  apparent.  To 
test  for  its  effect,  a series  of  runs  were  made  where  all 
values  were  held  constant  except  for  the  covariance.  The 
results  of  this  test  are  shown  in  Table  4-3.  These  results 
indicated  that  the  covariance  could  be  taken  to  be  zero  with- 
out effecting  the  calculated  probability  of  hit.  Although 
small  changes  in  the  probability  of  hit  were  computed, 
these  changes  were  well  within  the  sampling  noise  of  the 
model  and  could  be  considered  statistically  insignificant. 

One  point  worth  noting  is  that  the  P^  estimates  seem 
to  vary  more  for  negative  covariances  than  for  positive 
covariances,  especially  for  the  higher  correlation  coeffi- 
cients. If  it  is  felt  that  the  covariance  should  have  more 
impact  on  P,  estimates,  then  further  study  might  be  required 
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Table  4-3.  Effects  of  Aiming  Angle 

Covariance  on  Pk  Estimates 


Correlation 

Coefficient 

Covariance 

pk 

Confidence 

level 

.3333 

.00001 

.050 

.037 

— 

.065 

.0333 

.000001 

.049 

.036 

- 

.064 

.0167 

.0000005 

.048 

.035 

- 

.063 

0 

0 

.047 

.034 

- 

.062 

-.0167 

-.0000005 

.050 

.037 

- 

.065 

-.0333 

-.000001 

.051 

.038 

- 

.066 

-.3333 

-.00001 

.040 

.029 

- 

.054 

Sample  Size.  The  choice  of  sample  size  was  determined 
primarily  by  computation  time  considerations.  One  additional 
consideration  was  that  the  model  be  able  to  differentiate 
between  estimates  for  the  probability  of  hit  for  the  target 
vulnerable  areas  that  would  be  used.  A sample  size  of  1000 
was  taken  as  the  standard  to  be  used  since  this  was  the 
minimum  number  that  could  detect  probabilities  in  the  range 
of  .001.  As  mentioned  earlier,  sample  size  is  the  driving 
factor  determining  accuracy  of  predictions,  and  since  ab- 
solute accuracy  was  not  the  purpose  of  this  study,  sample 
sizes  of  1000  proved  adequate. 

When  using  Monte  Carlo  techniques,  the  absolute  values 
obtained  are  highly  influenced  by  the  seed  used  to  initiate 
the  random  number  generator.  Since  only  relative  values  were 
required  for  this  analysis,  a constant  value  was  selected  for 
the  starting  seed.  This  same  seed  was  used  to  initiate  each 
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replication.  By  using  the  same  seed,  effects  attributable  to 
the  random  number  stream  could  be  eliminated,  or  at  least 
pathologies  would  be  reflected  in  all  cases.  The  starting 
seed  used  in  this  study  was  350. 

This  section  has  described  the  variables  that  were  used 
in  AAASIM,  and  where  appropriate,  given  the  values  of  the 
variables  and  the  reasoning  behind  their  selections.  The 
next  section  will  present  the  results  of  the  analysis. 


Results  of  Sensitivity  Analysis 

The  purpose  of  this  analysis  is  to  examine  what  happens 
to  probability  of  hit  estimates  when  the  aiming  angle  var- 
iances are  varied.  It  should  be  noted  that  what  is  being 
varied  is  the  variances  of  the  aiming  angles  and  not  their 
standard  deviations.  If  this  fact  is  not  kept  in  mind  con- 
fusion in  interpreting  the  results  could  occur.  For  example, 
an  order  of  magnitude  change  in  the  variance  from  .00001  to 
.0001  is  only  a three  fold  or  so  change  in  the  standard 
deviations.  The  factors  that  are  going  to  effect  the  proba- 
bility of  hit  estimates  in  this  analysis  are  the  variances 
of  the  aiming  angles,  the  vulnerable  area  (presented  area  in 
this  case)  of  the  target,  the  range  to  the  intercept  point, 
and  the  ballistic  error  sources.  The  first  step  in  the 
analysis  will  be  to  look  at  the  effect  of  variance  changes 
on  the  probability  of  hit  estimates  when  all  ballistic  error 
sources  are  assigned  a value  of  zero. 


48 


assigned  values  of  zero,  the  intercept  range  was  set  at  1500 
meters,  the  target  elevation  was  set  to  zero  degrees,  and 
the  presented  area  of  the  target  was  62.8  meters2. 

A comment  is  in  order  at  this  point  on  the  references 
to  the  presented  area  of  the  target.  The  target  aircraft  is 
modelled  as  an  ellipsoid,  where  the  dimensions  of  the  semi- 
axes are  input  values  to  the  model.  The  simulated  anti- 
aircraft shell  actually  sees  a volume  rather  than  an  area. 

The  semi-axes  dimensions  input  for  this  series  of  runs  were: 

10  meters  for  the  X’  semi-axis,  and  2 meters  for  both  the  Y’ 
and  Z'  axes.  Viewed  along  a line  which  is  perpendicular  to 
the  X’  axis,  the  two-dimensional  area  presented  is  62.8  m2. 
This  presented  area  is  used  in  the  analysis  discussion  as  a 
convenience,  although  the  actual  volume  was  used  in  the 
simulation  runs. 

The  curve  shown  in  Figure  4-1,  for  zero  ballistic  errors, 
shows  a very  steep  slope  in  the  lower  regions  of  variances, 
this  slope  decreases  as  the  value  of  the  variance  increases. 
The  extremes  are  obvious,  when  the  variances  are  zero  the 
probability  of  hit  (in  the  zero  mean  error  case)  will  be  1.0, 
and  as  the  variance  increases  the  probability  of  hit  will 
tend  toward  zero.  The  important  point  to  notice  is  the 
value  of  the  slope  in  the  region  of  the  P001  computed  aim- 
ing angle  variance  (for  these  conditions),  .000045.  Looking 
at  this  condition,  a slight  increase  in  the  variance  will 
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Aiming  Angle  Variance  - (XI 0-5) 

Figure  4-1  . Comparison  of  Sensitivity  Response 
With  and  Without  Ballistic  Error  S 


produce  a small  decrease  in  the  probability  of  kill  (P^) 
while  an  equal  size  decrease  in  the  variance  will  produce  a 
larger  change  in  the  P^. 

The  key  point  in  applying  this  analysis  is  the  location 
of  the  current  aiming  angle  variance.  Its  value  will  deter- 
mine where  on  the  response  curve  the  user  is  working,  and 
therefore  the  magnitude  of  the  P^  change  for  a given  change 
in  the  variance  value.  The  next  logical  step  is  to  add 
values  for  the  ballistic  error  sources. 

Sensitivity  With  Ballistic  Errors.  The  response  curve 
on  Figure  4-1,  labeled  "With  Ballistic  Error  Sources",  was 
computed,  under  the  same  conditions  as  the  first  response 
curve  except  that  nominal  values  were  assigned  to  all  of  the 
ballistic  error  sources.  The  values  assigned  are  listed  in 
Table  3-1 . The  result  of  the  ballistic  error  sources  ad- 
dition was  to  decrease  the  overall  Pk  estimates,  and  more 
important,  the  response  curve  is  much  flatter. 

The  obvious  result  is  that  the  addition  of  the  ballistic 
error  sources  reduces  the  sensitivity  of  the  P^  estimates  to 
changes  in  the  aiming  angle  variances.  An  effect  not  so 
obvious  is  that  the  resulting  decrease  in  sensitivity  is 
also  a function  of  the  magnitude  of  the  P^.  estimates. 

Magnitude  of  P^  Estimates.  The  ballistic  error 
sources  and  the  errors  in  the  aiming  angles  tend  to  cause  a 
dispersion  in  the  fired  shells.  Therefore,  as  the  intercept 
range  increases  (all  other  model  values  held  constant)  the 
resulting  P^  estimates  will  decrease.  This  result  is  shown 
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AIMING  ANGLE  VARIANCE  - (XIO-^) 

Figure  4-2.  Affect  of  Magnitude  on  Pk  Rate  of  Change 
With  Aiming  Angle  Variance 


in  Figure  4-2,  note  the  change  in  scale  from  the  previous 
figure.  The  only  difference  in  the  series  of  runs  depicted 
in  Figure  4-2  is  the  increase  in  the  intercept  range.  The 
shell  dispersion  caused  hy  the  increase  range  resulted  in  a 
decrease  in  the  P^  estimates  computed.  It  is  clear,  from 
this  series  of  runs,  that  as  the  magnitude  of  the  Pk  esti- 
mates decreases  the  response  curves  tend  to  flatten. 

The  conclusion  to  he  drawn  from  this  part  of  the  analysis 
is  that  the  sensitivity  of  the  P^  estimates  is  a function  of 
the  magnitude  of  the  P-^  estimates  themselves. 

Effect  of  Ballistic  Error  Values.  The  ballistic  error 
sources  and  the  aiming  angle  variances  both  contribute  to 
the  dispersion  of  the  fired  projectile.  All  of  these 


sources  of  dispersion  are  combined  by  Monte  Carlo  techniques. 
Two  factors  that  will  determine  the  sensitivity  of  the  com- 
puted  P,  estimates  to  any  one  of  the  error  sources  are:  the 
sum  of  the  ballistic  error  variances  and  the  magnitude  of  the 
aiming  angle  variances.  As  the  sum  of  the  ballistic  error 
sources  variance  increases,  the  overall  effect  of  a change  in 
the  aiming  angle  variance  will  decrease.  Likewise,  if  the 
sum  of  the  ballistic  error  variances  is  significantly 
larger  than  the  aiming  variances,  the  effect  of  a change  in 
the  value  of  the  aiming  angle  variances  will  have  less  of  an 
impact  on  the  computed  Pk  estimates. 

These  two  factors  point  to  a critical  point  in  this 
study,  the  ballistic  error  sources  and  their  values.  The 
values,  as  well  as  the  sources,  were  those  currently  used 


Table  4-4.  Changes  in  P^  Estimates  for  a Decrease 
In  Variance  Prom  .00004  To  .00002 


Range 

P^  for  Variance 

TTOTO 7VUUU2 

Pk 

Increase 

500 

.301 

.351 

+ .050 

1500 

.035 

.064 

+ .029 

2000 

.018 

.037 

+ .019 

3000 

.013 

.017 

+ .004 

in  P001 . If  the  number  of  ballistic  sources  or  their  assumed 
■values  are  invalid,  the  results  of  this  analysis  may  change. 
The  soirees,  themselves,  are  fairly  well  established,  the 
values  may  be  questioned  but  are  not  significantly  suspect. 

Significance  of  Results.  The  significance  of  the  re- 
sults of  this  analysis  is  that  the  sensitivity  of  Pk  esti- 
mates is  relative  to  the  circumstances  of  the  engagement. 
Whether  P^  estimates  are  sensitive,  or  not,  to  changes  in 
the  aiming  angle  variances  will  depend  on  the  particular 
response  curve  in  question  (defined  by  the  conditions  of. the 
engagement)  and  the  location  on  the  response  curve  where  the 
current  aiming  angle  variance  is  positioned. 

Various  changes  in  Pk  estimates  that  could  be  expected 
for  a decrease  in  the  aiming  angle  variances  from  .00004  to 
.00002  are  listed  in  Table  4-4.  A decrease  in  aiming  angle 
variance  of  this  magnitude  equates  to  an  angular  standard 
deviation  change  of  about  2 milliradians . The  values  in 


Table  4-4  were  extracted  from  Figure  4-2. 

Referring  back  to  Figure  4-2,  the  region  of  the  response 
curve  with  the  largest  slope  occurs  below  a variance  of  about 
.00005.  Computed  aiming  angle  variances  from  AMRD’s  modified 
version  of  P001  indicates  that  the  current  value  of  the  var- 
iance is  about  .000045.  This  means  that  AMRL  is  presently- 
working  near  the  critical  point  of  the  response  curve. 
Decreases  in  the  variance  would  produce  much  more  of  a 
change  in  P^  estimates  than  an  equal  increase  in  the  var- 
iance. The  same  type  of  analysis  can  be  made  for  different 
ranges  (or  different  vulnerable  areas  since  both  result  in  a 
decrease  in  the  P^  estimate  values),  as  the  range  increases 
the  slope  of  the  response  curve  decreases.  Thus  equal 
changes  in  the  aiming  angle  variance  will  produce  decreasing 
changes  in  the  P^  estimates  as  the  range  of  intercept  in- 
creases. 

Confidence  Intervals.  Model  AAASIM  also  computes  a 
confidence  interval  for  each  P^  estimate  computed.  Figure 
4-3  illustrates  a print  out  from  AAASIM.  At  the  bottom  of 
the  figure  the  confidence  interval  is  shown  for  the  com- 
puted P^.  The  level  of  significance  for  the  confidence  in- 
terval is  an  input  to  the  model  and  can  be  changed  prior  to 
each  run,  if  so  desired. 

This  confidence  interval  reflects  the  sampling  noise 
level  of  the  model  and  should  be  used  to  compare  P^  estimates 
from  simular  runs.  This  noise  level  is  a direct  function  of 
the  sample  size  and  the  order  of  magnitude  of  the  P^.  estimate. 


--■■4.,  ui.  -.jjJL.  
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To  decrease  the  noise  level  of  the  model  requires  an  increase 
in  the  sample  size.  Sample  sizes  for  appropriate  degrees  of 
accuracy  of  predictions  are  contained  Tables  4-1  and  4-2. 


The  next  section  will  compare  the  results  from  AAASIM 
and  P001 . The  region  of  comparison  will  be  the  response 
curves  generated  from  each  model,  for  like  conditions. 

P001  Response  Curves 

A modified  version  of  P001  is  presently  used  by  AMR! 
to  generate  Pk  estimates  using  a more  sophisticated  track- 
ing model  than  the  original  P001  version.  In  this  section  a 
comparison  is  made  between  the  response  curves  of  P001  and 
the  response  curves  of  AAASIM.  Prior  to  making  this  com- 
parison, some  observations  will  be  made  concerning  assump- 
tions used  in  P001  and  the  modelling  methods  that  differ 
from  AAASIM.  One  of  the  basic  assumptions  used  in  P001  is 
that  the  shell  distribution  on  the  encounter  plane  is  bi- 
variate normal.  This  assumption  was  tested  in  this  study. 

Bivariate  Normal  Assumption.  Model  P001  uses  analytical 
methods  to  compute  the  Pk  estimates  for  a particular  run. 

To  make  these  computations  the  assumption  is  made  that  the 
shell  distribution  on  the  encounter  plane  is  bivariate 
normal.  If  the  encounter  plane  is  perpendicular  to  the  mean 
shell  velocity  and  the  aiming  distribution  is  bivariate 
normal,  the  shell  distribution  on  the  encounter  plane  can 
still  be  non-normal  due  to  non-linear  propagation  of  the 
shell.  This  situation  is  complicated  further  since  the 
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encounter  plane  is  actually  perpendicular  to  the  relative 
velocity  vector  of  the  shell  and  aircraft  at  the  time  of 
impact. 

Model  AAASIM  was  used  to  test  this  assumption  hy  gen- 
erating a shell  distribution  on  the  encounter  plane  and  test- 
ing this  distribution  for  normality.  The  method  of  testing 
is  described  in  Chapter  III  and  will  not  be  described  again 
here.  The  results  of  the  test  indicated  that  the  distribution 
passed  a Chi-square  goodness  of  fit  test  when  all  of  the  bal- 
listic error  sources  had  their  nominal  values.  A test  run 
where  the  ballistic  errors  were  assigned  values  of  zero  did 
not  pass  the  goodness  of  fit  test.  The  result  of  this  test 
is  shown  in  Figure  4-4.  Several  tests  were  run  at  various 
ranges,  elevations,  and  azimuth  angles  and  all  of  the  dis- 
tributions passed  the  tests  except  for  the  test  shown  in 
Figure  4-4. 

The  conclusion  to  be  drawn  is  the  normality  assumption 
is  valid  as  long  as  all  of  the  ballistic  error  sources  are 
assigned  values  near  their  nominal  values.  Erroneous  re- 
sults could  occur  if  P001  runs  were  made  considering  only 
the  errors  in  the  aiming  angles. 

Gravity  Drop  Effects.  Gravity  drop  was  not  considered 
in  P001.  One  of  the  objectives  of  this  study  was  to  test 
the  assumption  that  the  effects  of  gravity  drop  on  P^ 
estimates  could  be  neglected.  Gravity  drop  actually  causes 
a flatter  distribution  than  one  without  gravity  drop.  How- 


DISi-vISUriOM  0"  G J E l L IMPACT 

pours 

?3rinu  3 U7ErCc°T  PLANE  P^'5F'm:ULAK 

PELUIVE  VE 

. 33 I TY 

gf.a;tty  t'.oc  csnstdered 

InUT  P A(?AMCTFRS 

TI*»E  0r  FI'-E 

29.59‘3JCOO 

SECS, 

c A M ° L E ? 1 7 E 

1G  00 

i-rT«ii|u  ajvjnr  ;:jr,Lt  STD.?rV. 

. 2 B 5 '♦  7 7 2 

RADIAN'* 

ELFVAUON  AIMING  ANGLE  STr.TFy, 

•C05+772 

RADIANS 

a7Ty.|Tw  aiMNf-  ANGLE  VARIANCE 

.3033300 

FL rV AT Tu V AIDING  ANGLE  VARIANT 

. 0 0 u 0 3 0 0 

AIMING  l NHL*  COVARIANCE 

C. 00 33 C00 

mU77LE  VELOCITY 

S30.DP93GC0 

M/SFC 

.4.|ttLct  VrLOCI'Y  ST0.0F7. 

a. coo j coo 

M/SrC 

BALLISTIC  3ISCEK3I0N  STS. DEV. 

o. ro  to c oo 

RADIANS 

AT^C^o^ERlC  C7?Tir?ANCE  ST3.3EV. 

0. 3030000 

5 A 3 1 ANS 

g«jn  jit" fp  crr.c ev. 

o.inoioco 

radian^ 

FLIGHT  i» GUGHN F Sc  COEFFICIENT 

0.3030000 

«tl‘l  THE 0-: E T 1 0 A i_  3MFACT  POI.JT  < *^T I P ) 

’'IMF  FL IGM”  GP  SMELL 

. S 1 7 3 C Id 

SECS. 

TIMC  Qt  INTEVCEP7 

POSITION  op  a IF  CRAFT  A C *111° 

33.0003019 

SECS. 

Y-COOFOIN7TE 

• DO  0 3556 

Y-COOt H1NATE 

35’+.  3 000  3 00 

7-CCOPClNATE 

354.303u000 

POSITION  OF  SHELL  AT  N T X =* 

Y-POOPTiNATF 

. 30  3 3?  56 

Y-CQOFOINf  TE 

353.233* 571 

■'-'’ooroif  ATE 

35'* .1139240 

RANGE  CF  SMEL1 

500.57l073i 

NET  E»S 

CH  FL  L/ A I R CP A F T HIS7  ERROR  AT  MTIP 

• 1 “ 3 5 2 H 

IETF?? 

■i  1 T f A7Tm(ITw  ANGLE 

1. >737953 

RADIANS 

M7I3  ELEVATIO?  ANGLE 

515  47 

RADIANS 

RELATIVE  SPEEC  AT  MTIP 

752.529734? 

M/SFC 

0!FTRI'’'iTICN  c A * A ‘■’ETERS 

VERTICAL  MEAN 

. 232184 

METERS 

vertical  cTO.TEV. 

2.7375905 

METERS 

H0-.I7CiNAL  MEAN 

-.11973  3*5 

METERS 

MQPI  70MTAL  STn.CEV. 

1. 9731*0 3 

METERS 

V ERT  /HPR  7 POSITION  COVARTJMCE 

-1.2533675 

00r° EL ' T ION  C C EFeICI EMT 

-.2325242 

GD90NESS  OP  PIT  TC  NORMAL  OISFRI7 JTION 

NHmRvR  OF  CATAGORY  CELLS 

20 

VF ~r TC ^ L C4A-E0UA-E  STATISTT? 

If.  0.9303000 

VERTICAL  SIGNIFICANCE  LE7E. 

.or.  toc  cn 

HO^I7CNTAL  CHI-UUAAE  5T^TISrIC 

127. 2 ♦11C 00 

MQPI  70f.T  AL  SIGNIFICANCE  LFVFL 

•jju^COO 

Figure  4-4.  Normality  Test  of  Shell  Distribution 
Without  Ballistic  Error  Sources 

59 


i.e.  less  than  2000  meters.  Simulation  runs  of  AAASIM  that 


generated  a shell  distribution  showed  a negligible  effect  of 
gravity  drop  on  the  distribution.  Figures  4-5  and  4-6  show 
identical  runs  except  that  Figure  4-5  does  not  consider 
gravity  and  Figure  4-6  does  consider  gravity.  The  dif- 
ferences between  both  the  means  and  the  variances  of  these 
two  runs  is  in  the  second  decimal  place. 

Gravity  effects  were  also  tested  to  determine  the 
effect  on  P^  computations.  The  method  of  testing  was  to 
change  the  elevation  of  the  target  aircraft.  As  discussed 
in  Chapter  III,  the  effects  of  gravity  will  be  greatest 
(on  direction)  with  an  elevation  angle  of  zero  and  smallest 
with  an  elevation  angle  of  90°.  This  test  consisted  of  two 
sets  of  runs:  one  at  an  elevation  angle  of  zero  degrees,  and 
one  at  45°  elevation.  The  result  of  this  test  is  shown  in 
Figure  4-7.  The  different  response  curves  are  well  within 
the  confidence  interval  of  each  other.  Thus  a definite  con- 
clusion can  not  be  made,  except  to  note  the  trend.  The 
direction  of  the  trend  is  intuitive  since  the  zero  elevation 
run  had  overall  lower  P^  estimates  and  a steeper  slope. 

Zero  elevation  should  have  the  greatest  effect  on  the  shell 
distribution,  thus  causing  the  lower  P^  estimates.  The  ef- 
fects of  gravity  appear  to  be  negligible  in  the  intercept 
ranges  of  interest  in  this  study. 

The  final  observation  to  be  made  concerning  P001  is  the 
use  of  the  empirical  relationship  to  represent  the  shell 
trajectory,  versus  the  point  mass  differential  equation 
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Figure  4—6.  Shell  Distribution  With  Gravity  Effects 
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AIMING  ANGLE  VARIANCE  - (XIO"5) 

Figure  4“7.  Probability  of  Kill  vs.  Aiming  Angle  Variance 

Range  1500  meters 
Gravity  Drop  Effects 


i 


:! 


I 


, 


Table  4-5.  Comparison  of  Slant  Range  Calculations 
Between  P001  and  AAASIM 


Time  AAASIM  P001 


.5  sec 

436  m 

435 

1.0 

794 

793 

1.5 

1095 

1094 

2.0 

1353 

1354 

2.5 

1578 

1583 

3.0 

1777 

1787 

3.5 

I960 

1973 

4.0 

2130 

2144 

4.5 

2289 

2301 

5.0 

2439 

2452 

5.5 

2579 

2594 

6.0 

2716 

2729 

method  of  AAASIM. 

Enroirioal  Trajectory  Effects.  The  empirical  relation- 
ship P001  used  to  calculate  the  position  of  the  shell  versus 
time  is  given  in  Chapter  III.  This  relationship  compared 
quite  favorable  with  the  more  exact  method  used  by  AAASIM. 

A trajectory  slant  range  comparison  is  shown  in  Table  4-5. 
The  maximum  difference  between  the  two  models  was  15  meters, 
for  ranges  up  to  approximately  2700  meters. 

With  the  above  observations  in  mind,  a comparison  was 
made  between  the  modified  version  of  P001  and  AAASIM,  look- 
ing specifically  at  the  sensitivity  each  model  showed  to 
changes  in  the  aiming  angle  variances. 
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Sensitivity  Comparison.  The  final  step  in  this  study 
was  to  compare  the  sensitivity  obtained  from  the  modified 
version  of  P001  and  AAASIM.  To  put  the  test  in  perspective 
with  the  normal  results  of  P001 , the  vulnerable  area  was 
set  in  both  models  at  5.32  meters2.  This  value  was  select- 
ed from  the  P001  vulnerable  area  table  for  the  angle  of  the 
relative  velocity  vector.  Using  this  vulnerable  area,  an 
intercept  range  of  1500  meters,  an  elevation  angle  of  zero 
degrees,  and  a firing  azimuth  of  90°,  a series  of  runs  were 
made  with  each  model.  The  results  of  these  runs  are  shown 
in  Pigure  4-8. 

The  specific  point  to  notice  from  Pigure  4-8,  is  the 

« 

difference  in  the  response  curve  slopes.  The  response 
curve  from  P001  is  much  flatter  and  smoother,  while  the 
response  curve  from  AAASIM  shows  more  sensitivity  ot  the 
aiming  angle  variance  (in  the  range  of  variances  below 
about  .00008). 

The  reason  for  the  different  sensitivities  is  pure 
conjecture  at  this  point.  However,  since  the  normality 
assumption  proved  valid,  and  the  empirical  trajectory 
relationship  tested  very  close  to  the  trajectory  methods  of 
AAASIM,  it  would  appear  that  the  smoothing  dome  in  P001  is 
in  the  method  ot  target  representation.  It  should  be  not- 
ed that  the  difference  in  sensitivities  may  be  the  result 
of  hidden  factors,  such  as,  sampling  errors  in  AAASIM. 

It  should  also  be  noted  that  in  region  of  the  response 
curves  where  the  current  aiming  angle  variance  is  located 
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(.000045  computed  from  P001 ) the  P^  estimates  are  also  close, 
within  the  95%  confidence  interval  computed  for  the  data  of 
AAASIM. 

Observing  the  data  in  Figure  4-8,  it  would  appear 
feasible  that  more  sensitivity  could  be  obtained  from 
AAASIM  than  from  P001  for  otherwise  identical  cases. 

Whether  the  difference  in  sensitivity  is  significant , or 
not,  would  be  up  to  the  user.  A summary  of  the  above  analysis 
along  with  conclusions  and  recommendations  will  be  presented 
in  Chapter  Y. 
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V Conclusions  and  Recommendations 


This  study  has  concentrated  on  analyzing  the  sensi- 
tivity of  probability  of  kill  estimates,  derived  from  anti- 
aircraft artillery  and  aircraft  engagements,  to  parameters 
of  a supplied  aiming  angle  distribution.  The  supplied  dis- 
tribution was  assumed  to  be  bivariate  normal  with  zero  means. 
The  study  was  sponsored  by  AMR1  because  preliminary  analysis 
of  the  tracking  performance  of  human  trackers  has  shown  that 
significant  variations  in  tracking  capabilities  did  not  seem 
to  produce  equally  significant  variations  in  the  probability 
of  kill  estimates. 

In  the  process  of  AMRl’s  analysis,  a more  sophisticated 
tracking  submodel  than  the  original  P001  version  was  devel- 
oped. This  improved  submodel  was  incorporated  into  P001 
and  the  remaining  portions  of  P001  were  used  to  calculate  the 
probability  of  kill  estimates.  When  the  resulting  estimates 
did  not  show  the  expected  degree  of  response  to  tracking 
variations  the  possibility  existed  that  lack  of  precision  in 
the  remaining  submodels  of  P001  could  be  producing  a smooth- 
ing effect.  Three  areas  of  concern  were  identified  for 
further  study:  the  fire  control  director  submodel,  the 
exterior  ballistics  submodel,  and  the  terminal  effects  sub- 
model. The  exterior  and  terminal  ballistics  submodels  are 
the  point  of  this  study.  The  fire  control  director  submodel 
is  the  subject  of  another  study. 

In  order  to  calculate  probability  of  kill  estimates, 
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given  an  aiming  angle  distribution,  a dynamic  model  (AAASIM) 
was  developed  that  used  point  mass  equations  of  motion  to 
calculate  the  trajectory  of  the  shell,  and  an  ellipsoidal 
shaped  target  was  used  to  model  the  aircraft.  Monte  Carlo 
sampling  techniques  were  used  to  sample  the  random  errors 
from  the  aiming  angle  distribution  and  the  ballistic  error 
sources. 

The  analysis  methods  used  in  this  study  and  the  re- 
sults of  the  analysis  have  been  presented  in  previous 
chapters.  This  chapter  will  discuss  the  conclusions  that 
can  be  drawn  from  the  analysis  and  provide  some  recommenda- 
tions for  improving  the  response  of  P001 . 

Conclusions 

The  first  point  to  be  made  concerning  conclusions  that 
can  be  drawn  from  this  analysis  is  that,  by  design,  the 
analysis  was  very  narrow  in  scope.  The  purpose  of  the  study 
was  to  concentrate  specifically  on  the  sensitivity  of  the 
probability  of  kill  estimates  to  the  variances  of  the  aiming 
angle  distribution.  To  accomplish  this  purpose,  many  var- 
iables in  the  model  were  fixed  in  value.  The  values  used 
were  believed  to  be  reasonable  and  consistent  with  the  real 
world  system;  however,  if  these  constant  values  should  change 
the  literal  output  from  AAASIM  will  change.  By  limiting  the 
scope  of  the  analysis  the  desired  sensitivity  could  be  pin- 
pointed, but  the  literal  output  is  of  little  value.  The 
real  value  of  the  model  is  the  relative  results  obtained 


and  the  trends  that  developed. 

Probability  of  Kill  Sensitivity.  The  most  interesting 
heuristic  result  of  this  analysis  was  the  shape  of  the  re- 
sponse curve  obtained  when  Pk  was  plotted  versus  the  aiming 
angle  variance.  The  response  curves  consistently  showed  two 
distinctive  slopes.  Initially,  when  the  aiming  angle  var- 
iances was  low,  the  curves  had  a relatively  steep  slope. 

This  initial  slope  remained  fairly  constant  until  the  aiming 
variance  increased  to  a range  roughly  equavalent  to  the  sum 
of  the  variances  of  the  ballistic  error  sources.  At  this 
;•  point  the  response  curves  tended  to  flatten  to  a shallow 

slope.  See  Figure  4-8,  which  is  included  here  again  for 
convenience. 

The  interesting  range  of  aiming  angle  variances  occurred 
between  0.0  and  .00020.  After  the  latter  value,  the  re- 
sponse curve  sloped  gradually  downward,  eventually  resulting 
in  P^  values  very  near  zero.  Relatively  sensitivity  will 
then  be  highly  dependent  upon  the  position  of  the  nominal 
value  of  the  variance.  If  the  nominal  aiming  angle  variance 
is  on  the  steeper  portion  of  the  curve,  the  sensitivity  of 
Pk  estimates  will  considerably  more  than  if  the  nominal 
value  is  on  the  flatter  portion  of  the  curve. 

The  factors  that  appeared  to  drive  the  Pk  sensitivity 
were:  the  magnitude  of  the  Pk  values  themselves,  the  vul- 
nerable area  of  the  target,  and  the  sum  of  the  ballistic 
error  sources.  The  affect  of  these  factors  can  be  seen 
from  a simple  analytical  model. 
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AIMING  ANGLE  VARIANCE  - (X10  J) 

Figure  4-8.  Probability  of  Kill  vs.  Aiming  Angle  Variance 

Range  1500  meters 
Comparison  of  P001  and  AAASIM 


Using  the  assumption  that  the  shell  distribution  oh 
the  encounter  plane  is  bivariate  normal,  a simple  analogy- 
can  be  drawn  using  a one  dimensional  example.  In  this  one 
dimensional  case  the  P^.  can  be  calculated  by  the  following 
relationship: 


r\ 


(2H)*(Sa 


SS)* 


X2 


R 


exp(-i  ^ (5-D 


where:  P^  - is  the  probability  of  kill 

- is  the  aiming  angle  variance 
S|  - is  the  sum  of  the  ballistic  error  variances 
X - is  the  vulnerable  area  expressed  in  radians 
K - is  the  length  of  semi-axis  of  the  target 
R - is  the  range  to  the  target 


The  response  curves  that  were  developed  in  this  study  are 
plots  of  P^  versus  aiming  angle  variance . The  slope  of 
these  curves  is  the  partial  derivative  of  the  P^  with  re- 
spect to  the  aiming  single  variance.  For  the  above  analogy 
this  partial  is: 


<*  Sf 


S1+S2 


2 -(S2+S2)exp(-*  — S ) 

R S^+S* 

(5-2) 


where  all  variables  are  the  same  as  in  equation  5-1 . 
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Analyzing  the  terms  in  equation  5-2,  for  the  one 
dimensional  case,  provides  some  indications  of  the  factors 


that  will  affect  the  slope  of  the  response  curves.  The 
first  term  proves  to  he  the  most  significant,  it  shows 
that  the  slope  will  become  more  negative  as  the  magnitude 
of  the  P^  increases  (for  a constant  denominator)  and  alter- 
nately, as  the  denominator  increases  in  magnitude  the 
slope  will  decrease.  If  the  magnitude  of  the  ballistic 
error  variances  remains  constant,  then  the  slope  decreases 
as  the  aiming  angle  variance  increases.  This  corresponds 
directly  to  the  observed  findings  of  this  study. 

The  second  term  is  a function  of  the  vulnerable  area 

TT 

(^)  as  well  as  the  aiming  angle  variance  and  the  variance  of 
the  ballistic  error  sources.  As  the  vulnerable  area  in- 
creases, the  slope  will  become  more  negative  (have  a steep- 
er slope).  An  increase  in  the  vulnerable  area  will  also 
increase  the  P^  calculated  from  equation  5-1 . This  analysis 
shows  the  impact  of  the  vulnerable  area  on  the  shape  of  the 
response  curves. 

The  above  analogy  provides  some  analytical  interpretation 
for  the  findings  of  this  study  with  the  exception  of  the 
apparent  flattening  of  the  response  curves  generated  in 
this  study.  This  apparent  flattening  could  very  well  be 
caused  by  sampling  error  since  it  occurred  in  areas  of  low 
P^  and  small  perturbations  in  the  number  of  hits  in  this 
region  could  cause  apparent  abnormal  behavior  in  the  re- 
:.o nse  curves. 
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To  make  a more  accurate  determination  of  the  exact 
shape  of  the  response  curves  generated  by  AAASIM,  several 
replications  should  be  made  for  each  point  on  the  curves. 
The  values  from  these  replications  could  then  be  averaged 
and  the  mean  of  the  points  plotted.  Since  this  was  not 
possible  in  this  study  (because  of  time  constraints)  it 
can  not  be  ascertained  unequivocally  that  AAASIM  produces 
more  sensitivity  than  P001 . However,  it  does  appear  from 
the  trends  of  the  response  curves  produced  by  AAASIM,  that 
the  methods  employed  by  P001  could  be  dampening  some  of 
the  sensitivity. 

The  more  important  result  of  this  analysis  has  been 
the  highlighting  of  driving  factors  in  determining  the 
sensitivity  of  P^  estimates  to  aiming  angle  variances, 
and  the  realization  that  on  some  portions  of  the  response 
curves  the  sensitivity  is  very  low. 

Covariance  Affects.  The  covariance  of  the  aiming 
angle  distribution  was  tested  to  ascertain  what  impact  it 
had  on  the  P^  estimates.  The  covariance  was  varied  plus 
and  minus  three  orders  of  magnitude,  using  a nominal  value 
of  the  variance  of  .00003.  The  results  of  this  test  in- 
dicated that  the  P^  range  for  the  plus  covariances  was  bet- 
ween .047  and  .050,  and  the  minus  covariances  was  .047  and 
.040.  Although  the  actual  values  are  not  statistically 
significant,  there  appears  to  be  a trend  indicating  more 
affect  for  negative  covariances  than  for  positive  covar- 
inaces . 
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The  conclusion  drawn  in  this  analysis,  is  that,  gener- 
ally the  covariance  can  he  ignored  in  the  P^  calculations; 
however,  if  the  covariance  is  negative  and  on  the  same  order 
of  magnitude  of  the  aiming  angle  variance  further  study  of 
this  effect  might  he  required.  The  implicit  assumption  used, 
hy  taking  the  covariance  term  to  he  zero,  is  that  the  aiming 
angle  errors  are  independent. 

Normality  Test.  Model  P001  calculates  P^.  estimates 
using  the  assumption  that  the  shell  distribution  on  the 
encounter  plane  is  bivariate  normal.  One  of  the  objectives 
of  this  study  was  to  test  this  assumption.  The  assumption 
was  tested  and  found  to  he  valid.  Over  the  range  of  aiming 
angle  variances  tested,  the  shell  distribution  passed  a 
Chi-square  goodness  of  fit  test  for  all  values  of  the  var- 
iances. This  test  was  made  using  all  of  the  ballistic  error 
sources  set  at  their  nominal  values.  One  exception  to  the 
above  results  occurred  when  the  shell  distribution  was  deter- 
mined without  the  ballistic  error  sources,  this  case  did  not 
the  goodness  of  fit  test. 

The  conclusion  that  can  be  drawn  is  that  the  ballistic 
error  sources,  in  conjunction  with  the  aiming  angle  errors, 
produce  a central  limit  effect  on  the  distribution  resulting 
in  a distribution  which  is  normal.  Using  a model,  such  as 
P001 , without  the  ballistic  error  sources  would  produce  a 
shell  distribution  that  was  not  normal  and  thus  could  in- 
validate the  results. 
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Gravity  Affects.  The  model  developed  for  this  study- 
included  the  affect  of  gravity  in  the  trajectory  calculations 
Two  tests  were  conducted  to  ascertain  the  affect  of  gravity 


on  the  Pk  estimates  calculated.  The  first  test  used  ident- 
ical conditions,  except  for  gravity,  at  an  elevation  angle 
of  zero.  The  response  curves  generated  from  these  two  runs 
overlapped.  The  second  test  examined  the  case  where  one  run 
was  made  at  an  elevation  of  0°  and  another  run  at  45°,  all 
other  conditions  the  same.  The  difference  in  the  two  re- 
sponse curves  was  still  insignificant;  however,  the  run  at 
45°  showed  a possible  trend  of  increased  Pk  estimates. 

It  is  conceivable  that  at  extended  ranges  the  affects 
of  gravity  could  be  significant,  but  for  the  ranges  consider- 
ed in  this  study  the  affects  of  gravity  can  be  ignored. 

Trajectory  Calculations.  A comparison  was  made  bet- 
ween the  slant  range  of  the  shell  calculated  by  AAASIM,  using 
the  more  exact  trajectory  model,  and  the  slant  range  cal- 
culated by  P001  using  its  empirical  relationship.  Por  slant 
ranges  up  to  2700  meters,  the  results  of  the  P001  empirical 
relationship  was  found  to  be  within  15  meters  of  results  of 
AAASIM.  Considering  tha  fact  that  this  is  a slant  range  and 
not  a miss  distance,  the  differences  between  the  two  methods 
is  considered  to  be  insignificant. 

This  conclusion,  in  addition  to  the  conclusion  on 
gravity  affects,  verifies  that  POOI’s  empirical  trajectory 
model  is  adequate  for  the  intercept  ranges  considered  valid 
for  this  study. 


In  light  of  the  conclusions  drawn  above,  some  recom- 
mendations can  now  be  made  on  possible  modifications  to 
P001  and  areas  requiring  further  study. 

Recommendations 

One  of  the  big  drawbacks  to  AAASIM  was  the  computer 
time  required  to  run  a sample.  The  majority  of  the  com- 
putation time  was  required  by  ODERT  to  solve  the  differential 
equations  of  the  trajectory.  The  results  of  the  analysis 
indicate  that  this  time  could  be  saved  by  using  the 
empirical  relationship  of  P001  (or  a simular  one)  to  cal- 
culate the  trajectory  of  the  shell. 

Additionally,  it  appears  that  the  difference  between 
the  response  curves  generated  by  P001  and  AAASIM  is  caused 
by  the  method  of  sampling  the  error  sources  and/or  the 
method  of  representing  the  target.  A logical  extention 
of  this  study  would  be  to  modify  P001  to  use  Monte  Carlo 
sampling  techniques  and  a shaped  target  target. 

The  vulnerable  area  of  the  target  could  be  varied  by 
internally  adjusting  the  dimensions  of  the  shaped  target 
using  the  vulnerable  area  values  obtained  from  the  already 
incorporated  vulnerable  area  table.  The  Monte  Carlo  sampl- 
ing would  replace  the  present  expected  value  method  used  by 
P001 . Another  alternative  would  be  to  use  a systematic 
sampling  method  in  place  of  the  Monte  Carlo  method. 

The  resulting  modification  would  obviously  take  longer 
to  run  than  the  present  version  of  P001 , but  not  as  long  as 


AAASIM.  The  obvious  tradeoff  would  be  whether  the  more 
accurate  response  that  could  possibly  be  obtained  would 
be  worth  the  additional  computation  time  to  the  user. 

Overall  the  results  obtained  from  P001  compare  quite 
favorably  with  the  results  from  AAASIM,  and  it  can  not  be 
determined  with  any  degree  of  certainty  that  the  modelling 
methods  of  AAASIM  are  any  better  than  P001 . Obviously, 
the  operational  efficiency  of  P001  is  far  superior  to  that 
of  AAASIM  and  any  gain  in  sensitivity  would  be  costly  in 
computation  time. 
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INITIALIZE  COUNTERS  FOR  DISTRIBUTION  PARAMETERS 
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SUBKOUTINE  NTEST(KATS»STATZfSTATX) 

DIMENSION  X<1000>  ,7(10  00)  ,COVAR<3>  ,0UM(3>  ,CA( 2,2) ,B(3)  t HKAREA  (6)  » 
1C (3) »OBSC (20)»CSOBS ( 20) , STAT (3)  , ARRAY ( 1 0 0 0) , STATZ C 31 ,STATX(3) 
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